Jul 09 09:54:20.177234 ip-10-0-138-234 systemd[1]: Starting Kubernetes Kubelet... Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.576675 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581249 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581261 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581265 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581268 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581271 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581274 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581276 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:20.641662 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581281 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581283 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581286 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581288 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581291 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581293 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581296 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581298 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581301 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581304 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581306 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581312 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581315 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581317 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581320 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581325 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581330 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581333 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581336 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:20.644373 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581339 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581342 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581344 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581347 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581350 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581352 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581355 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581368 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581372 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581375 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581377 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581380 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581383 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581385 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581388 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581390 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581393 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581395 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581399 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:20.646037 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581404 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581407 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581411 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581414 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581417 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581420 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581422 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581424 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581427 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581430 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581433 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581435 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581438 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581441 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581443 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581445 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581448 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581451 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581454 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581456 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:20.646982 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581459 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581461 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581464 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581467 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581470 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581472 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581475 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581477 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581480 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581483 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581486 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581488 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581491 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581494 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581497 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581500 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581503 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581505 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581508 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581510 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:20.647984 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581513 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581515 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581518 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581520 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581524 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581527 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581530 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581533 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581536 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581538 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581541 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581543 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581546 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.581548 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583483 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583492 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583496 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583499 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583503 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:20.649084 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583505 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583508 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583512 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583514 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583517 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583519 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583522 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583525 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583528 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583530 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583533 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583536 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583538 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583541 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583543 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583546 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583548 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583551 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583554 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583557 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:20.650190 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583559 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583562 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583565 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583568 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583570 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583573 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583575 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583578 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583581 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583584 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583586 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583589 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583592 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583594 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583597 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583599 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583602 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583604 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583607 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583610 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:20.651270 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583612 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583615 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583618 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583620 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583623 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583625 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583628 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583631 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583634 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583636 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583639 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583642 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583646 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583649 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583652 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583655 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583658 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583660 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583664 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583667 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:20.652133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583669 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583672 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583675 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583678 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583681 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583683 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583686 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583689 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583691 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583694 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583697 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583700 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583703 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583705 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583708 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583711 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583713 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583715 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583718 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583721 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:20.653210 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583724 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:20.653975 ip-10-0-138-234 systemd[1]: Started Kubernetes Kubelet. Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583727 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583731 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583734 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583736 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583739 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583741 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583745 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583749 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583751 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583754 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583756 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583759 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.583763 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584191 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584197 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584200 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584204 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584207 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:20.654236 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584209 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584212 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584216 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584218 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584221 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584224 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584227 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584230 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584232 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584238 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584241 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584244 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584247 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584249 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584252 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584254 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584258 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584262 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584264 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584267 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:20.655490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584272 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584274 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584277 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584280 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584283 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584285 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584288 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584290 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584293 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584296 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584298 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584301 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584303 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584306 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584308 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584311 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584313 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584316 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584319 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584322 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:20.656634 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584324 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584328 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584330 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584333 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584335 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584338 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584340 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584343 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584345 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584348 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584350 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584353 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584357 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584359 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584363 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584366 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584370 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584372 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584375 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584377 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:20.657822 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584380 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584383 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584386 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584388 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584391 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584393 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584396 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584399 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584401 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584404 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584406 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584409 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584412 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584417 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584420 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584423 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584426 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584429 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584432 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:20.659026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584435 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584439 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584442 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584445 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584447 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584451 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584453 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584456 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584458 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584461 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584463 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584466 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584468 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584471 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.584473 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584533 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584540 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584545 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584550 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584553 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584556 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584560 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 09:54:20.660131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584564 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584567 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584570 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584574 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584579 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584582 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584585 2576 flags.go:64] FLAG: --cgroup-root="" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584588 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584590 2576 flags.go:64] FLAG: --client-ca-file="" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584593 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584596 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584601 2576 flags.go:64] FLAG: --cluster-domain="" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584603 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584606 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584609 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584613 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584621 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584625 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584628 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584631 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584634 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584637 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584640 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584644 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 09:54:20.661143 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584646 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584650 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584653 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584656 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584659 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584662 2576 flags.go:64] FLAG: --enable-server="true" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584665 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584669 2576 flags.go:64] FLAG: --event-burst="100" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584671 2576 flags.go:64] FLAG: --event-qps="50" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584675 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584678 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584681 2576 flags.go:64] FLAG: --eviction-hard="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584686 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584689 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584692 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584695 2576 flags.go:64] FLAG: --eviction-soft="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584698 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584701 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584704 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584707 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584709 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584712 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584715 2576 flags.go:64] FLAG: --feature-gates="" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584720 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584723 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 09:54:20.662231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584726 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584729 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584732 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584735 2576 flags.go:64] FLAG: --help="false" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584738 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584741 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584752 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584755 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584758 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584762 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584765 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584767 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584770 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584773 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584776 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584779 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584782 2576 flags.go:64] FLAG: --kube-reserved="" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584785 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584787 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584792 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584795 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584798 2576 flags.go:64] FLAG: --lock-file="" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584801 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584804 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 09:54:20.663519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584806 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584812 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584814 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584817 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584820 2576 flags.go:64] FLAG: --logging-format="text" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584823 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584827 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584830 2576 flags.go:64] FLAG: --manifest-url="" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584833 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584837 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584840 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584843 2576 flags.go:64] FLAG: --max-pods="110" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584846 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584850 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584852 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584855 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584858 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584861 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584864 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584870 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584873 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584876 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584879 2576 flags.go:64] FLAG: --pod-cidr="" Jul 09 09:54:20.666913 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584882 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584887 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584890 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584893 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584897 2576 flags.go:64] FLAG: --port="10250" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584900 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584904 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe8ae01e063315ec" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584907 2576 flags.go:64] FLAG: --qos-reserved="" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584910 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584913 2576 flags.go:64] FLAG: --register-node="true" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584916 2576 flags.go:64] FLAG: --register-with-taints="" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584919 2576 flags.go:64] FLAG: --registry-burst="10" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584922 2576 flags.go:64] FLAG: --registry-qps="5" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584924 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584927 2576 flags.go:64] FLAG: --reserved-memory="" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584931 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584935 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584938 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584955 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584958 2576 flags.go:64] FLAG: --runonce="false" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584960 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584963 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584966 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584969 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584972 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584975 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 09:54:20.668218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584978 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584981 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584984 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584987 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584989 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584992 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584995 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.584998 2576 flags.go:64] FLAG: --system-cgroups="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585001 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585006 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585010 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585013 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585017 2576 flags.go:64] FLAG: --tls-min-version="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585020 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585023 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585026 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585029 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585031 2576 flags.go:64] FLAG: --v="2" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585035 2576 flags.go:64] FLAG: --version="false" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585039 2576 flags.go:64] FLAG: --vmodule="" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585043 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.585046 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585158 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585163 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:20.852057 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585166 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585169 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585171 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585174 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585176 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585181 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585184 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585187 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585190 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585193 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585195 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585198 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585200 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585203 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585205 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585208 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585210 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585213 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585217 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585220 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:20.868937 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585222 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585225 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585232 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585234 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585237 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585240 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585242 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585244 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585247 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585249 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585253 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585256 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585260 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585263 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585266 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585269 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585272 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585274 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585277 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585280 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:20.869661 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585282 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585285 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585287 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585290 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585292 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585295 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585297 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585300 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585302 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585305 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585308 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585311 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585313 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585316 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585318 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585321 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585323 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585326 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585328 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:20.870370 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585331 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585333 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585336 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585340 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585342 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585345 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585347 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585350 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585352 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585355 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585368 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585373 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585377 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585380 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585383 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585386 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585389 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585392 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585395 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585397 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:20.870961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585400 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585403 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585405 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585409 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585412 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585415 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585417 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585420 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585423 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585426 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585428 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585431 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585434 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585436 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585439 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585442 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585445 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:20.871547 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.585448 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.586190 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.594342 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.594357 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.594378 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.594383 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594437 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594442 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594446 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594449 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594452 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594456 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594459 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594462 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594466 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:20.872034 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594470 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594473 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594475 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594478 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594481 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594483 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594486 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594488 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594491 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594494 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594497 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594499 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594502 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594505 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594508 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594511 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594514 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594516 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594519 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594522 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:20.872490 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594525 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594527 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594530 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594533 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594535 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594540 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594543 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594545 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594548 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594550 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594553 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594556 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594558 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594561 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594563 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594566 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594568 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594571 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594574 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594576 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:20.873063 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594579 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594582 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594584 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594587 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594590 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594593 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594596 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594598 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594601 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594604 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594608 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594611 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594616 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594619 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594621 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594625 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594627 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594631 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594634 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:20.873649 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594637 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594639 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594642 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594645 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594647 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594650 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594652 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594655 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594658 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594660 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594663 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594666 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594669 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594671 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594674 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594677 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594679 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594682 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594685 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:20.874224 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594687 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594690 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594692 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594695 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594697 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594700 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594703 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594706 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594708 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594712 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594715 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594720 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.594725 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594834 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:20.874773 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594839 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594843 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594846 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594849 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594852 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594854 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594857 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594860 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594862 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594865 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594868 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594871 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594875 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594878 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594881 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594884 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594887 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594889 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594892 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:20.875187 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594894 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594897 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594900 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594903 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594905 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594909 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594911 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594914 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594917 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594920 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594922 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594926 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594928 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594931 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594934 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594936 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594955 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594959 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594961 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594964 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:20.875730 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594967 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594970 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594973 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594975 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594978 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594982 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594984 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594987 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594990 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594992 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594995 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.594997 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595000 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595003 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595005 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595008 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595010 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595014 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595017 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595019 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:20.876281 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595022 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595024 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595027 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595031 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595034 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595036 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595039 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595041 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595044 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595047 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595049 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595051 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595054 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595056 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595059 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595061 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595064 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595066 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595070 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595072 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:20.876855 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595075 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595077 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595080 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595083 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595085 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595088 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595090 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595093 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595095 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595099 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595101 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595105 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595108 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595111 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595114 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595118 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595120 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595122 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:20.877426 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:20.595125 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.595130 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.595231 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.597414 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.598678 2576 server.go:1019] "Starting client certificate rotation" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.598776 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.598815 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.601016 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.604849 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.619915 2576 log.go:25] "Validated CRI v1 runtime API" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.622013 2576 log.go:25] "Validated CRI v1 image API" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.622777 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 09:54:20.877975 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.625540 2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 893c0a29-5867-4803-a74c-2e323dd93789:/dev/nvme0n1p4 a86fc9a3-dec1-44ae-a883-e5f78ba9ac20:/dev/nvme0n1p3] Jul 09 09:54:20.878335 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.625558 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 09:54:20.878335 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.633135 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.633722 2576 manager.go:216] Machine: {Timestamp:2026-07-09 09:54:20.632179369 +0000 UTC m=+0.339480375 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3080024 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2892119ce5b880ad2f30d938ac34bb SystemUUID:ec289211-9ce5-b880-ad2f-30d938ac34bb BootID:290967fe-91a2-4e30-9d58-2821f148ac09 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0d:de:00:65:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0d:de:00:65:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:df:b7:a0:10:c8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.633851 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.633958 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.636694 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.636728 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-234.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.637540 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.637554 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.637565 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.638396 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.639844 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.642840 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643009 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643044 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643056 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643078 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643093 2576 kubelet.go:398] "Adding apiserver pod source" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643105 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 09:54:20.878761 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643520 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.643546 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.645631 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.647551 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.647590 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649526 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649544 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649553 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649560 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649570 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649576 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649585 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649590 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649597 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649603 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649612 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.649617 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.650330 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.650337 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.653420 2576 server.go:1268] "Started kubelet" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.653759 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.655222 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.656102 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.656134 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.657512 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.659850 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-95kpw" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.663353 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.663666 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.666013 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.666479 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.666838 2576 factory.go:55] Registering systemd factory Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.666850 2576 factory.go:223] Registration of the systemd container factory successfully Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667014 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667045 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667050 2576 factory.go:153] Registering CRI-O factory Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667059 2576 factory.go:223] Registration of the crio container factory successfully Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667015 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667109 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667134 2576 factory.go:103] Registering Raw factory Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667146 2576 manager.go:1196] Started watching for new ooms in manager Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.667765 2576 manager.go:349] Starting recovery of all containers Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.668098 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.668111 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.668969 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-95kpw" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.676458 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-234.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.676505 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 09:54:20.879408 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.676546 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.676746 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.676625 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-234.ec2.internal.18c0967019e11025 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-234.ec2.internal,UID:ip-10-0-138-234.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-234.ec2.internal,},FirstTimestamp:2026-07-09 09:54:20.653400101 +0000 UTC m=+0.360701107,LastTimestamp:2026-07-09 09:54:20.653400101 +0000 UTC m=+0.360701107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-234.ec2.internal,}" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.685796 2576 manager.go:354] Recovery completed Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.690254 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.690375 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697376 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697408 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697424 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697891 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697896 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697902 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.697928 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.698623 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.698646 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.698656 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.699135 2576 policy_none.go:49] "None policy: Start" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.699148 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.699157 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.700024 2576 policy_none.go:47] "Start" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.742692 2576 manager.go:337] "Starting Device Plugin manager" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.742838 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.742851 2576 server.go:87] "Starting device plugin registration server" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.743065 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.743075 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.743270 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.743331 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.743338 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.743723 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.743771 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.813511 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.814571 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.814610 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.814629 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.814659 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.817593 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.843655 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.845187 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.845208 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.845226 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.845251 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.880727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.854478 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.882098 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.854494 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-234.ec2.internal\": node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:20.882098 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.868699 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.914966 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"] Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.915035 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.916555 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.916577 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.916585 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.917655 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.917769 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.917794 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.918288 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.918294 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.918315 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.918325 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.918314 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.918393 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.919372 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.919394 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.919976 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.919996 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.920007 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.951480 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-234.ec2.internal\" not found" node="ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.955750 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-234.ec2.internal\" not found" node="ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:20.969518 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.969569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/590b685de265f07f8c0aacb999f9d120-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"590b685de265f07f8c0aacb999f9d120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.969590 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/590b685de265f07f8c0aacb999f9d120-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"590b685de265f07f8c0aacb999f9d120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:20.998539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:20.969608 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be7f2ab963a045a1608548ebc092ac92-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"be7f2ab963a045a1608548ebc092ac92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:21.069735 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:21.069762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/590b685de265f07f8c0aacb999f9d120-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"590b685de265f07f8c0aacb999f9d120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:21.069846 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/590b685de265f07f8c0aacb999f9d120-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"590b685de265f07f8c0aacb999f9d120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:21.070156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/590b685de265f07f8c0aacb999f9d120-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"590b685de265f07f8c0aacb999f9d120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:21.070172 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/590b685de265f07f8c0aacb999f9d120-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"590b685de265f07f8c0aacb999f9d120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:21.070174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be7f2ab963a045a1608548ebc092ac92-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"be7f2ab963a045a1608548ebc092ac92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:21.070194 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be7f2ab963a045a1608548ebc092ac92-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"be7f2ab963a045a1608548ebc092ac92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" Jul 09 09:54:21.225475 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:21.170351 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:22.954607 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:22.763745 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:22.954607 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:22.767782 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" Jul 09 09:54:22.954607 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:22.779636 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:22.954607 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:22.880012 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.170591 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:22.980448 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.170591 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:23.080795 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.170591 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.106913 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 09:54:23.170591 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.107104 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 09:54:23.170591 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.107112 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:23.181690 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.185491 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 09:49:22 +0000 UTC" deadline="2028-03-21 00:19:45.338350996 +0000 UTC" Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.185887 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14894h25m22.15264794s" Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.233747 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:23.282417 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:23.292252 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590b685de265f07f8c0aacb999f9d120.slice/crio-ddb1ea01d6f29cf5bb1dd5b8c28e5713d39be47d25ab565ed2f587efc3547a6b WatchSource:0}: Error finding container ddb1ea01d6f29cf5bb1dd5b8c28e5713d39be47d25ab565ed2f587efc3547a6b: Status 404 returned error can't find the container with id ddb1ea01d6f29cf5bb1dd5b8c28e5713d39be47d25ab565ed2f587efc3547a6b Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.297300 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.327420 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" event={"ID":"be7f2ab963a045a1608548ebc092ac92","Type":"ContainerStarted","Data":"cc88b677f792d3302a28dc95f4e5e5a469f21494067bbfbbd88a2d1842933550"} Jul 09 09:54:23.379519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.328372 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"590b685de265f07f8c0aacb999f9d120","Type":"ContainerStarted","Data":"ddb1ea01d6f29cf5bb1dd5b8c28e5713d39be47d25ab565ed2f587efc3547a6b"} Jul 09 09:54:23.383101 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:23.382652 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.588940 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:23.483232 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found" Jul 09 09:54:23.588940 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.503104 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:23.588940 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.576387 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" Jul 09 09:54:23.588940 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.588419 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 09:54:23.594060 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.589300 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" Jul 09 09:54:23.788937 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.596831 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 09:54:24.089131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:23.995622 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.153272 2576 apiserver.go:52] "Watching apiserver" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.161264 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.161723 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-wvb64","openshift-multus/network-metrics-daemon-tlvqc","openshift-network-diagnostics/network-check-target-xdxhj","openshift-network-operator/iptables-alerter-8dd7b","openshift-ovn-kubernetes/ovnkube-node-l7cws","kube-system/global-pull-secret-syncer-hpl6x","kube-system/konnectivity-agent-8prlt","kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qnftd","openshift-image-registry/node-ca-dvxlb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal","openshift-multus/multus-additional-cni-plugins-b2vq9","openshift-multus/multus-pcpcr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp"] Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.163139 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.164191 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.165235 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.165323 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.165729 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.165735 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.165734 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5wlx7\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.166460 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.166540 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.166989 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.167081 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.167194 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.167323 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v9q5z\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.167730 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.169016 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.169321 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h886w\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.169574 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.169661 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.169768 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.169656 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.170234 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.174397 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.175709 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.175854 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.175923 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.176810 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-csfhq\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.177066 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8rnsd\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.177070 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 09:54:24.195137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.177523 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.178430 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.178634 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.180606 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.180888 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.181907 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.181967 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.183357 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.184485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.184861 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.184909 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.184999 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z2js2\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.185005 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.185649 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.185804 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.185896 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5l75b\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186093 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186114 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186762 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-os-release\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186790 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-multus-certs\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186815 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186867 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-cni-netd\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186873 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186891 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42h8\" (UniqueName: \"kubernetes.io/projected/d153a1ca-b6ce-4f28-b715-0b183167826b-kube-api-access-t42h8\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186916 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/049c3668-cc57-448f-85bf-5f99de584f31-kubelet-config\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysctl-conf\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.196692 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-sys\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186994 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.186996 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-cni-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-systemd\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187115 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-ovnkube-config\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0115514f-42d4-40a8-94ea-cfb3069fa696-hosts-file\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187183 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-system-cni-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187231 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-cnibin\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-node-log\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-kubelet\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187337 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-ovn\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-netns\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysctl-d\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187396 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187429 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n4brl\"" Jul 09 09:54:24.197926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57ef2190-5fd3-412e-890b-2059714507e6-cni-binary-copy\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q4nb\" (UniqueName: \"kubernetes.io/projected/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-kube-api-access-4q4nb\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-hostroot\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-conf-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187562 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/57ef2190-5fd3-412e-890b-2059714507e6-multus-daemon-config\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-slash\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187600 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-run-netns\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0acb4e7b-437f-47da-a57a-de6383ee9625-tmp\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187642 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14190e32-666c-4490-b533-adafc8c34b41-konnectivity-ca\") pod \"konnectivity-agent-8prlt\" (UID: \"14190e32-666c-4490-b533-adafc8c34b41\") " pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-tuned\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-run\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-lib-modules\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187732 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14190e32-666c-4490-b533-adafc8c34b41-agent-certs\") pod \"konnectivity-agent-8prlt\" (UID: \"14190e32-666c-4490-b533-adafc8c34b41\") " pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187762 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-iptables-alerter-script\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187788 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-cni-bin\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/049c3668-cc57-448f-85bf-5f99de584f31-dbus\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-kubernetes\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.198790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115514f-42d4-40a8-94ea-cfb3069fa696-tmp-dir\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187892 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-kubelet\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-etc-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-socket-dir-parent\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.187975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-modprobe-d\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188032 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-host-slash\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-k8s-cni-cncf-io\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188083 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-env-overrides\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysconfig\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188128 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-host\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-etc-kubernetes\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188196 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-systemd-units\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188214 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-log-socket\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbj6r\" (UniqueName: \"kubernetes.io/projected/0115514f-42d4-40a8-94ea-cfb3069fa696-kube-api-access-nbj6r\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-ovnkube-script-lib\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188314 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-var-lib-kubelet\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188335 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-cni-bin\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.199630 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-cni-multus\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-var-lib-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188437 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-run-ovn-kubernetes\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188465 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d153a1ca-b6ce-4f28-b715-0b183167826b-ovn-node-metrics-cert\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188484 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6lzr\" (UniqueName: \"kubernetes.io/projected/57ef2190-5fd3-412e-890b-2059714507e6-kube-api-access-k6lzr\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-systemd\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188558 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188555 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5q9k\" (UniqueName: \"kubernetes.io/projected/0acb4e7b-437f-47da-a57a-de6383ee9625-kube-api-access-s5q9k\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188965 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.188998 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w4h5c\"" Jul 09 09:54:24.200592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.191218 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 09:54:24.209883 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.209818 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hkr22" Jul 09 09:54:24.218202 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.218173 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hkr22" Jul 09 09:54:24.277434 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.277408 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 09:54:24.288801 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.288921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288807 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-ovnkube-config\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.288921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0115514f-42d4-40a8-94ea-cfb3069fa696-hosts-file\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.288921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-sys-fs\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.288921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.288921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288891 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-serviceca\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.288921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288911 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288916 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0115514f-42d4-40a8-94ea-cfb3069fa696-hosts-file\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288946 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.288975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nf7n\" (UniqueName: \"kubernetes.io/projected/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-kube-api-access-8nf7n\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-device-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-system-cni-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289103 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-cnibin\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289126 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-node-log\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7df5\" (UniqueName: \"kubernetes.io/projected/e8e8a28e-d99d-4e19-86be-f23e804f0830-kube-api-access-x7df5\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289166 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-system-cni-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289184 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-kubelet\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289222 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-cnibin\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289234 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-ovn\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289252 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-node-log\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-system-cni-dir\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289292 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-ovn\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289294 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-kubelet\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289320 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-netns\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysctl-d\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289406 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-netns\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289417 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-ovnkube-config\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289477 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysctl-d\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289505 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57ef2190-5fd3-412e-890b-2059714507e6-cni-binary-copy\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q4nb\" (UniqueName: \"kubernetes.io/projected/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-kube-api-access-4q4nb\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.289565 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:24.289606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-etc-selinux\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289603 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-hostroot\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289626 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-conf-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/57ef2190-5fd3-412e-890b-2059714507e6-multus-daemon-config\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289661 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-slash\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.289695 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:24.789675427 +0000 UTC m=+2.987767202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289791 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-conf-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-run-netns\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0acb4e7b-437f-47da-a57a-de6383ee9625-tmp\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289868 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-slash\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14190e32-666c-4490-b533-adafc8c34b41-konnectivity-ca\") pod \"konnectivity-agent-8prlt\" (UID: \"14190e32-666c-4490-b533-adafc8c34b41\") " pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289906 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-tuned\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289918 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-run-netns\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-run\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-lib-modules\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14190e32-666c-4490-b533-adafc8c34b41-agent-certs\") pod \"konnectivity-agent-8prlt\" (UID: \"14190e32-666c-4490-b533-adafc8c34b41\") " pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57ef2190-5fd3-412e-890b-2059714507e6-cni-binary-copy\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.290433 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.289996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-iptables-alerter-script\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290025 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l27l5\" (UniqueName: \"kubernetes.io/projected/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-kube-api-access-l27l5\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290061 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-cni-binary-copy\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290113 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-run\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290113 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-cni-bin\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290171 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/049c3668-cc57-448f-85bf-5f99de584f31-dbus\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290171 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290182 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-cni-bin\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-kubernetes\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290199 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-lib-modules\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115514f-42d4-40a8-94ea-cfb3069fa696-tmp-dir\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-socket-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290239 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-hostroot\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290339 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-kubelet\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290372 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-etc-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.291254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290394 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-socket-dir-parent\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290416 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-modprobe-d\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290433 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14190e32-666c-4490-b533-adafc8c34b41-konnectivity-ca\") pod \"konnectivity-agent-8prlt\" (UID: \"14190e32-666c-4490-b533-adafc8c34b41\") " pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290485 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-kubelet\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290517 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-etc-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290548 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-socket-dir-parent\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290603 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-kubernetes\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290603 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-modprobe-d\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290622 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-host-slash\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290654 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-cnibin\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290680 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-host-slash\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-k8s-cni-cncf-io\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290731 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/57ef2190-5fd3-412e-890b-2059714507e6-multus-daemon-config\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-env-overrides\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290759 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-iptables-alerter-script\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290792 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-k8s-cni-cncf-io\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115514f-42d4-40a8-94ea-cfb3069fa696-tmp-dir\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290833 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysconfig\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.291956 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290868 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/049c3668-cc57-448f-85bf-5f99de584f31-dbus\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290869 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-host\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290900 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysconfig\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290943 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-host\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.290976 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-host\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-etc-kubernetes\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-systemd-units\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291049 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-etc-kubernetes\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291076 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-systemd-units\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-log-socket\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291089 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-env-overrides\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbj6r\" (UniqueName: \"kubernetes.io/projected/0115514f-42d4-40a8-94ea-cfb3069fa696-kube-api-access-nbj6r\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-ovnkube-script-lib\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291127 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-log-socket\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291144 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-var-lib-kubelet\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291180 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-cni-bin\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-cni-multus\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-var-lib-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.292855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-run-ovn-kubernetes\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d153a1ca-b6ce-4f28-b715-0b183167826b-ovn-node-metrics-cert\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291287 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6lzr\" (UniqueName: \"kubernetes.io/projected/57ef2190-5fd3-412e-890b-2059714507e6-kube-api-access-k6lzr\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-systemd\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291325 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5q9k\" (UniqueName: \"kubernetes.io/projected/0acb4e7b-437f-47da-a57a-de6383ee9625-kube-api-access-s5q9k\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291349 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-registration-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwlm7\" (UniqueName: \"kubernetes.io/projected/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-kube-api-access-mwlm7\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-os-release\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-os-release\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291440 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-multus-certs\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291468 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-cni-netd\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42h8\" (UniqueName: \"kubernetes.io/projected/d153a1ca-b6ce-4f28-b715-0b183167826b-kube-api-access-t42h8\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291539 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/049c3668-cc57-448f-85bf-5f99de584f31-kubelet-config\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysctl-conf\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291583 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-sys\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291605 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-cni-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.294407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291619 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d153a1ca-b6ce-4f28-b715-0b183167826b-ovnkube-script-lib\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291626 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-systemd\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-cni-bin\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291643 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-var-lib-cni-multus\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291703 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-host-run-multus-certs\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291722 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-sys\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291733 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-var-lib-openvswitch\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291758 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-sysctl-conf\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291768 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-multus-cni-dir\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291770 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57ef2190-5fd3-412e-890b-2059714507e6-os-release\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291605 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-var-lib-kubelet\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291796 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-run-systemd\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291802 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-cni-netd\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291806 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d153a1ca-b6ce-4f28-b715-0b183167826b-host-run-ovn-kubernetes\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291854 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-systemd\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.291900 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/049c3668-cc57-448f-85bf-5f99de584f31-kubelet-config\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.293926 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0acb4e7b-437f-47da-a57a-de6383ee9625-etc-tuned\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.294059 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0acb4e7b-437f-47da-a57a-de6383ee9625-tmp\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.295273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.294426 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14190e32-666c-4490-b533-adafc8c34b41-agent-certs\") pod \"konnectivity-agent-8prlt\" (UID: \"14190e32-666c-4490-b533-adafc8c34b41\") " pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.296170 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.294584 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d153a1ca-b6ce-4f28-b715-0b183167826b-ovn-node-metrics-cert\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.299040 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.299022 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:24.299144 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.299043 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:24.299144 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.299057 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:24.299144 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.299110 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:24.799093342 +0000 UTC m=+2.997185143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:24.299757 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.299731 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q4nb\" (UniqueName: \"kubernetes.io/projected/b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe-kube-api-access-4q4nb\") pod \"iptables-alerter-8dd7b\" (UID: \"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe\") " pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.300926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.300901 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbj6r\" (UniqueName: \"kubernetes.io/projected/0115514f-42d4-40a8-94ea-cfb3069fa696-kube-api-access-nbj6r\") pod \"node-resolver-wvb64\" (UID: \"0115514f-42d4-40a8-94ea-cfb3069fa696\") " pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.301578 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.301543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42h8\" (UniqueName: \"kubernetes.io/projected/d153a1ca-b6ce-4f28-b715-0b183167826b-kube-api-access-t42h8\") pod \"ovnkube-node-l7cws\" (UID: \"d153a1ca-b6ce-4f28-b715-0b183167826b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.301931 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.301909 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6lzr\" (UniqueName: \"kubernetes.io/projected/57ef2190-5fd3-412e-890b-2059714507e6-kube-api-access-k6lzr\") pod \"multus-pcpcr\" (UID: \"57ef2190-5fd3-412e-890b-2059714507e6\") " pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.302852 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.302825 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5q9k\" (UniqueName: \"kubernetes.io/projected/0acb4e7b-437f-47da-a57a-de6383ee9625-kube-api-access-s5q9k\") pod \"tuned-qnftd\" (UID: \"0acb4e7b-437f-47da-a57a-de6383ee9625\") " pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.392249 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-sys-fs\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392249 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-serviceca\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.392249 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392241 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nf7n\" (UniqueName: \"kubernetes.io/projected/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-kube-api-access-8nf7n\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392267 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-device-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392291 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392298 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-sys-fs\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7df5\" (UniqueName: \"kubernetes.io/projected/e8e8a28e-d99d-4e19-86be-f23e804f0830-kube-api-access-x7df5\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392365 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-device-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-system-cni-dir\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392434 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-system-cni-dir\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392447 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-etc-selinux\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392506 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l27l5\" (UniqueName: \"kubernetes.io/projected/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-kube-api-access-l27l5\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392532 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-cni-binary-copy\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-socket-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-cnibin\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392668 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392669 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-etc-selinux\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392700 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-host\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.392722 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392741 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-registration-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392759 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-cnibin\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.392771 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:54:24.892758441 +0000 UTC m=+3.090850216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392774 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-socket-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwlm7\" (UniqueName: \"kubernetes.io/projected/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-kube-api-access-mwlm7\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392797 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-registration-dir\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.392916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392721 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-serviceca\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392803 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-os-release\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392894 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.392915 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.393012 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-host\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.393120 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8e8a28e-d99d-4e19-86be-f23e804f0830-os-release\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.393362 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-cni-binary-copy\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.393557 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.393376 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8e8a28e-d99d-4e19-86be-f23e804f0830-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.402302 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.402268 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwlm7\" (UniqueName: \"kubernetes.io/projected/09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4-kube-api-access-mwlm7\") pod \"aws-ebs-csi-driver-node-25hsp\" (UID: \"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.402581 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.402553 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7df5\" (UniqueName: \"kubernetes.io/projected/e8e8a28e-d99d-4e19-86be-f23e804f0830-kube-api-access-x7df5\") pod \"multus-additional-cni-plugins-b2vq9\" (UID: \"e8e8a28e-d99d-4e19-86be-f23e804f0830\") " pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.402672 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.402621 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nf7n\" (UniqueName: \"kubernetes.io/projected/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-kube-api-access-8nf7n\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.405658 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.405635 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l27l5\" (UniqueName: \"kubernetes.io/projected/3014cfa9-cf7d-4b35-bd09-ff468ba689a2-kube-api-access-l27l5\") pod \"node-ca-dvxlb\" (UID: \"3014cfa9-cf7d-4b35-bd09-ff468ba689a2\") " pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.480375 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.480345 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:24.489093 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.489071 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pcpcr" Jul 09 09:54:24.501945 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.501922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8dd7b" Jul 09 09:54:24.509466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.509447 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:24.516770 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.516752 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnftd" Jul 09 09:54:24.525185 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.525163 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wvb64" Jul 09 09:54:24.532818 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.532799 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvxlb" Jul 09 09:54:24.540258 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.540240 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" Jul 09 09:54:24.545819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.545800 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" Jul 09 09:54:24.796549 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.796509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:24.796762 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.796669 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:24.796762 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.796746 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:25.796728338 +0000 UTC m=+3.994820133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:24.897188 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.897130 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:24.897359 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:24.897223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:24.897359 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.897327 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:24.897463 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.897388 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:54:25.897367672 +0000 UTC m=+4.095459477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:24.897463 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.897325 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:24.897463 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.897424 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:24.897463 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.897436 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:24.897650 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:24.897466 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:25.897459225 +0000 UTC m=+4.095551000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:25.220145 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.220085 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:49:24 +0000 UTC" deadline="2028-01-11 10:09:32.848989226 +0000 UTC" Jul 09 09:54:25.220145 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.220139 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13224h15m7.628854572s" Jul 09 09:54:25.265722 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.265697 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acb4e7b_437f_47da_a57a_de6383ee9625.slice/crio-7f9e19edaae81c31b68045ca4bf4eb976f440509b1ee9416b4c31e8ee96a754a WatchSource:0}: Error finding container 7f9e19edaae81c31b68045ca4bf4eb976f440509b1ee9416b4c31e8ee96a754a: Status 404 returned error can't find the container with id 7f9e19edaae81c31b68045ca4bf4eb976f440509b1ee9416b4c31e8ee96a754a Jul 09 09:54:25.267453 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.267426 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3014cfa9_cf7d_4b35_bd09_ff468ba689a2.slice/crio-7392082330dde911fb3fc7ddd84b48a890dc5956cc04cb3d37a78be5977cb836 WatchSource:0}: Error finding container 7392082330dde911fb3fc7ddd84b48a890dc5956cc04cb3d37a78be5977cb836: Status 404 returned error can't find the container with id 7392082330dde911fb3fc7ddd84b48a890dc5956cc04cb3d37a78be5977cb836 Jul 09 09:54:25.270992 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.270971 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd153a1ca_b6ce_4f28_b715_0b183167826b.slice/crio-bac18c82a44c0dfb6041924d584db65b2c559e9ad5c72082582200a4a219e9fa WatchSource:0}: Error finding container bac18c82a44c0dfb6041924d584db65b2c559e9ad5c72082582200a4a219e9fa: Status 404 returned error can't find the container with id bac18c82a44c0dfb6041924d584db65b2c559e9ad5c72082582200a4a219e9fa Jul 09 09:54:25.271786 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.271764 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ef2190_5fd3_412e_890b_2059714507e6.slice/crio-56e297ee1777f1f61c39bca23e79cda9aa439b448eeaeaad7d1d20a52a8b6498 WatchSource:0}: Error finding container 56e297ee1777f1f61c39bca23e79cda9aa439b448eeaeaad7d1d20a52a8b6498: Status 404 returned error can't find the container with id 56e297ee1777f1f61c39bca23e79cda9aa439b448eeaeaad7d1d20a52a8b6498 Jul 09 09:54:25.272942 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.272898 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14190e32_666c_4490_b533_adafc8c34b41.slice/crio-fb34ea96d97159ca1c309fe6cdb39b1394c1c74638cecdea4671a19e92e45b0a WatchSource:0}: Error finding container fb34ea96d97159ca1c309fe6cdb39b1394c1c74638cecdea4671a19e92e45b0a: Status 404 returned error can't find the container with id fb34ea96d97159ca1c309fe6cdb39b1394c1c74638cecdea4671a19e92e45b0a Jul 09 09:54:25.274624 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.274605 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e8a28e_d99d_4e19_86be_f23e804f0830.slice/crio-1093dc55a43670c71b30c2a832d1b5d49ab1d2f42dab9a1e369c2702ba18c991 WatchSource:0}: Error finding container 1093dc55a43670c71b30c2a832d1b5d49ab1d2f42dab9a1e369c2702ba18c991: Status 404 returned error can't find the container with id 1093dc55a43670c71b30c2a832d1b5d49ab1d2f42dab9a1e369c2702ba18c991 Jul 09 09:54:25.275406 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.275383 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9fd0fb4_8bbd_45ed_98e7_4faef7ecddbe.slice/crio-6ce01fa15e2669598a88a67f8b53dd895ba195e732ea8e7b8feaee03aeb8b98f WatchSource:0}: Error finding container 6ce01fa15e2669598a88a67f8b53dd895ba195e732ea8e7b8feaee03aeb8b98f: Status 404 returned error can't find the container with id 6ce01fa15e2669598a88a67f8b53dd895ba195e732ea8e7b8feaee03aeb8b98f Jul 09 09:54:25.276077 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.275985 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0115514f_42d4_40a8_94ea_cfb3069fa696.slice/crio-c090e4f9a2f09067f637f37961125d6bd4e21391082091760ab00241601756dd WatchSource:0}: Error finding container c090e4f9a2f09067f637f37961125d6bd4e21391082091760ab00241601756dd: Status 404 returned error can't find the container with id c090e4f9a2f09067f637f37961125d6bd4e21391082091760ab00241601756dd Jul 09 09:54:25.278218 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:54:25.278137 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fb5f7b_3db9_4f3c_94c4_bc698d3b8de4.slice/crio-5b811aa3afd325bdfe77ec3880dfc746ba13567508f0a96f6db20c6881804b1b WatchSource:0}: Error finding container 5b811aa3afd325bdfe77ec3880dfc746ba13567508f0a96f6db20c6881804b1b: Status 404 returned error can't find the container with id 5b811aa3afd325bdfe77ec3880dfc746ba13567508f0a96f6db20c6881804b1b Jul 09 09:54:25.324116 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.323985 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:25.324259 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.324234 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:25.332538 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.332511 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" event={"ID":"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4","Type":"ContainerStarted","Data":"5b811aa3afd325bdfe77ec3880dfc746ba13567508f0a96f6db20c6881804b1b"} Jul 09 09:54:25.333626 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.333607 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerStarted","Data":"1093dc55a43670c71b30c2a832d1b5d49ab1d2f42dab9a1e369c2702ba18c991"} Jul 09 09:54:25.334738 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.334702 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8prlt" event={"ID":"14190e32-666c-4490-b533-adafc8c34b41","Type":"ContainerStarted","Data":"fb34ea96d97159ca1c309fe6cdb39b1394c1c74638cecdea4671a19e92e45b0a"} Jul 09 09:54:25.335807 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.335780 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvxlb" event={"ID":"3014cfa9-cf7d-4b35-bd09-ff468ba689a2","Type":"ContainerStarted","Data":"7392082330dde911fb3fc7ddd84b48a890dc5956cc04cb3d37a78be5977cb836"} Jul 09 09:54:25.336930 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.336910 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8dd7b" event={"ID":"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe","Type":"ContainerStarted","Data":"6ce01fa15e2669598a88a67f8b53dd895ba195e732ea8e7b8feaee03aeb8b98f"} Jul 09 09:54:25.338080 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.338051 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wvb64" event={"ID":"0115514f-42d4-40a8-94ea-cfb3069fa696","Type":"ContainerStarted","Data":"c090e4f9a2f09067f637f37961125d6bd4e21391082091760ab00241601756dd"} Jul 09 09:54:25.339132 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.339113 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pcpcr" event={"ID":"57ef2190-5fd3-412e-890b-2059714507e6","Type":"ContainerStarted","Data":"56e297ee1777f1f61c39bca23e79cda9aa439b448eeaeaad7d1d20a52a8b6498"} Jul 09 09:54:25.340257 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.340237 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"bac18c82a44c0dfb6041924d584db65b2c559e9ad5c72082582200a4a219e9fa"} Jul 09 09:54:25.341189 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.341170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnftd" event={"ID":"0acb4e7b-437f-47da-a57a-de6383ee9625","Type":"ContainerStarted","Data":"7f9e19edaae81c31b68045ca4bf4eb976f440509b1ee9416b4c31e8ee96a754a"} Jul 09 09:54:25.803797 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.803719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:25.803957 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.803891 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:25.803957 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.803952 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:27.803933753 +0000 UTC m=+6.002025534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:25.904731 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.904699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:25.904884 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:25.904768 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:25.904956 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.904881 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:25.904956 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.904942 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:54:27.904921515 +0000 UTC m=+6.103013292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:25.905366 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.905348 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:25.905443 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.905375 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:25.905443 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.905387 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:25.905443 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:25.905433 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:27.905418071 +0000 UTC m=+6.103509850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:26.221211 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.221174 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:49:24 +0000 UTC" deadline="2027-12-12 08:02:01.729489857 +0000 UTC" Jul 09 09:54:26.221211 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.221210 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12502h7m35.508283709s" Jul 09 09:54:26.324461 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.324281 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:26.324461 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:26.324416 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:26.326882 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.326857 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:26.326993 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:26.326958 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:26.360628 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.359466 2576 generic.go:356] "Generic (PLEG): container finished" podID="590b685de265f07f8c0aacb999f9d120" containerID="4f41c5b6a1ffd7cb2c2f1cbddf7eebc3f2f2d89635835868380164c271228a2b" exitCode=0 Jul 09 09:54:26.360628 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.359557 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"590b685de265f07f8c0aacb999f9d120","Type":"ContainerDied","Data":"4f41c5b6a1ffd7cb2c2f1cbddf7eebc3f2f2d89635835868380164c271228a2b"} Jul 09 09:54:26.367068 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:26.365815 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" event={"ID":"be7f2ab963a045a1608548ebc092ac92","Type":"ContainerStarted","Data":"fa51408a0c0fc665b6e8c698885619f052b59de8580e9023c53d6b3246e668ed"} Jul 09 09:54:27.324438 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.323998 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:27.324438 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.324090 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:27.397370 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.397098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"590b685de265f07f8c0aacb999f9d120","Type":"ContainerStarted","Data":"e50c7022df1cc095331339063db2b65b5ac06926a5979ea2753a32f6a124329d"} Jul 09 09:54:27.414981 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.414926 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" podStartSLOduration=4.414905019 podStartE2EDuration="4.414905019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:26.408388191 +0000 UTC m=+4.606479992" watchObservedRunningTime="2026-07-09 09:54:27.414905019 +0000 UTC m=+5.612996814" Jul 09 09:54:27.415188 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.415162 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" podStartSLOduration=4.415138426 podStartE2EDuration="4.415138426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:27.414596343 +0000 UTC m=+5.612688140" watchObservedRunningTime="2026-07-09 09:54:27.415138426 +0000 UTC m=+5.613230224" Jul 09 09:54:27.819134 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.819096 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:27.819397 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.819261 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:27.819397 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.819322 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:31.819302105 +0000 UTC m=+10.017393881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:27.919947 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.919865 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:27.919947 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:27.919944 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:27.920226 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.920028 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:27.920226 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.920038 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:27.920226 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.920048 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:27.920226 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.920062 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:27.920226 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.920095 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:54:31.92007704 +0000 UTC m=+10.118168832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:27.920226 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:27.920112 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:31.920103418 +0000 UTC m=+10.118195192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:28.325412 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:28.324866 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:28.325412 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:28.324987 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:28.325412 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:28.325255 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:28.325412 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:28.325373 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:29.325124 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:29.324638 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:29.325124 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:29.324778 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:30.324566 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:30.324522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:30.325025 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:30.324656 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:30.325025 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:30.324696 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:30.325025 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:30.324805 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:31.324796 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:31.324299 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:31.324796 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.324433 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:31.852860 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:31.852822 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:31.853038 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.852975 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:31.853103 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.853047 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:39.853028116 +0000 UTC m=+18.051119904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:31.953352 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:31.953319 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:31.953505 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:31.953395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:31.953505 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.953471 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:31.953505 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.953497 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:31.953673 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.953505 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:31.953673 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.953511 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:31.953673 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.953568 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:54:39.953548285 +0000 UTC m=+18.151640079 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:31.953673 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:31.953588 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:39.953577362 +0000 UTC m=+18.151669140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:32.325822 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:32.325376 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:32.325822 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:32.325461 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:32.325822 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:32.325485 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:32.325822 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:32.325583 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:33.324387 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:33.324355 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:33.324530 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:33.324475 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:34.324906 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:34.324569 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:34.324906 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:34.324729 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:34.325448 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:34.325087 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:34.325448 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:34.325199 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:35.324254 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:35.324224 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:35.324400 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:35.324324 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:36.324280 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:36.324242 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:36.324713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:36.324251 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:36.324713 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:36.324405 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:36.324713 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:36.324437 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:37.324940 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:37.324911 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:37.325360 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:37.325034 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:38.324785 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:38.324751 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:38.324785 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:38.324771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:38.325022 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:38.324874 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:38.325366 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:38.325009 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:39.324244 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:39.324211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:39.324445 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:39.324319 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:39.914519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:39.914482 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:39.914991 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:39.914646 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:39.914991 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:39.914707 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:55.914690294 +0000 UTC m=+34.112782082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:40.015388 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:40.015354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:40.015553 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:40.015426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:40.015553 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.015535 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:40.015637 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.015549 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:40.015637 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.015574 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:40.015637 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.015586 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:54:56.015572336 +0000 UTC m=+34.213664112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:40.015637 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.015588 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:40.015765 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.015642 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:56.01562565 +0000 UTC m=+34.213717430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:40.324912 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:40.324879 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:40.324912 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:40.324910 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:40.325173 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.325034 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:40.325224 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:40.325202 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:41.324235 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:41.324195 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:41.324644 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:41.324316 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:42.325094 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:42.325062 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:42.325610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:42.325178 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:42.325610 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:42.325184 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:42.325610 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:42.325264 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:43.324240 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:43.324204 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:43.324404 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:43.324332 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:44.325304 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.325089 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:44.325785 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.325306 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:44.325785 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:44.325394 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:44.325785 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:44.325543 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:44.431602 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.431574 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wvb64" event={"ID":"0115514f-42d4-40a8-94ea-cfb3069fa696","Type":"ContainerStarted","Data":"acf54d959b3c4325ccd4602b3f8ad472714958fc32d9816d00fa48326cc8e69e"} Jul 09 09:54:44.433162 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.433124 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pcpcr" event={"ID":"57ef2190-5fd3-412e-890b-2059714507e6","Type":"ContainerStarted","Data":"b6045fc00ec2e0551045608d791f45f6f6e6d8bb2d695e9f1a6f93bce020ea7a"} Jul 09 09:54:44.437111 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.437090 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"29e2a55abfd46f5f6160a691c8a5dcfeb81cf2e97e5154c71665caf645dea118"} Jul 09 09:54:44.437234 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.437145 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"2c786db98eaf95a4138bef0194b11c88e384f463e4a135fa904563204fd6f79f"} Jul 09 09:54:44.437234 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.437206 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"142fa295dad948a3a2d09deb564d1eb6a33337845a9fb0b38d90b172a203b470"} Jul 09 09:54:44.438510 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.438487 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnftd" event={"ID":"0acb4e7b-437f-47da-a57a-de6383ee9625","Type":"ContainerStarted","Data":"51bc00cdfad7a82379b8d5b9db80f28d59e54efdff34cd8aae9eea586215bf11"} Jul 09 09:54:44.439764 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.439746 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" event={"ID":"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4","Type":"ContainerStarted","Data":"2d18d483f99f8dd1f6fdd836fd63632c8e63e8b7a86093d62440588aab2d357e"} Jul 09 09:54:44.440949 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.440931 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8e8a28e-d99d-4e19-86be-f23e804f0830" containerID="6cb076feffc492fc7bd713eb60b0ceb0d20b3a65633a828f2e4b9b3f7a8b5540" exitCode=0 Jul 09 09:54:44.441001 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.440992 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerDied","Data":"6cb076feffc492fc7bd713eb60b0ceb0d20b3a65633a828f2e4b9b3f7a8b5540"} Jul 09 09:54:44.442394 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.442373 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8prlt" event={"ID":"14190e32-666c-4490-b533-adafc8c34b41","Type":"ContainerStarted","Data":"6944b0c8276a1b0ecbbef8fbcaef39981d4140faba07659c9e6aeeb4e4923d03"} Jul 09 09:54:44.443626 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.443604 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvxlb" event={"ID":"3014cfa9-cf7d-4b35-bd09-ff468ba689a2","Type":"ContainerStarted","Data":"475a668f65aeb096e6a3e239c4393d538e19afbd6009a8f8684005e9727911ba"} Jul 09 09:54:44.448937 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.448901 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wvb64" podStartSLOduration=3.7078046860000002 podStartE2EDuration="22.448890346s" totalImagesPullingTime="18.74108566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:44.448732864 +0000 UTC m=+22.646824660" watchObservedRunningTime="2026-07-09 09:54:44.448890346 +0000 UTC m=+22.646982143" Jul 09 09:54:44.490874 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.490833 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pcpcr" podStartSLOduration=3.688400884 podStartE2EDuration="22.490816778s" totalImagesPullingTime="18.802415894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:44.490293178 +0000 UTC m=+22.688384988" watchObservedRunningTime="2026-07-09 09:54:44.490816778 +0000 UTC m=+22.688908575" Jul 09 09:54:44.505539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.505500 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8prlt" podStartSLOduration=7.524708513 podStartE2EDuration="22.505489688s" totalImagesPullingTime="14.980781175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:44.505199499 +0000 UTC m=+22.703291296" watchObservedRunningTime="2026-07-09 09:54:44.505489688 +0000 UTC m=+22.703581486" Jul 09 09:54:44.520652 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.520620 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dvxlb" podStartSLOduration=3.773575633 podStartE2EDuration="22.52060947s" totalImagesPullingTime="18.747033837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:44.520527896 +0000 UTC m=+22.718619692" watchObservedRunningTime="2026-07-09 09:54:44.52060947 +0000 UTC m=+22.718701267" Jul 09 09:54:44.537972 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.537943 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qnftd" podStartSLOduration=3.772689884 podStartE2EDuration="22.537931741s" totalImagesPullingTime="18.765241857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:44.53771262 +0000 UTC m=+22.735804417" watchObservedRunningTime="2026-07-09 09:54:44.537931741 +0000 UTC m=+22.736023543" Jul 09 09:54:44.631746 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.631717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:44.718127 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:44.718100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:45.324350 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.324320 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:45.324589 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:45.324440 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:45.448700 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.448663 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8dd7b" event={"ID":"b9fd0fb4-8bbd-45ed-98e7-4faef7ecddbe","Type":"ContainerStarted","Data":"c5ed59374f69e9fb54a310a5721b7ac0b18ad284624ccc22ca82dbcb04c8911b"} Jul 09 09:54:45.451775 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.451749 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"d7a3244619457015cc3a92819510599e162a35808d51c9d2df7081bd4e733ec1"} Jul 09 09:54:45.451916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.451822 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"87be2f5d89e1ce56a979930bfa932d872a6fbe13cf4fc51072239de740795773"} Jul 09 09:54:45.451916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.451853 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"6d266711df2914b0a1e076dda280158cd1d842e7519be9d8d2c2d079e1564f89"} Jul 09 09:54:45.465108 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.465068 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8dd7b" podStartSLOduration=4.723272378 podStartE2EDuration="23.465056034s" totalImagesPullingTime="18.741783656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:45.464886205 +0000 UTC m=+23.662978007" watchObservedRunningTime="2026-07-09 09:54:45.465056034 +0000 UTC m=+23.663147831" Jul 09 09:54:45.587580 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.587463 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:45.651228 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:45.651179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:46.076397 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.076369 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 09:54:46.149137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.149103 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:46.231179 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.231130 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:46.263900 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.263822 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T09:54:46.076395381Z","UUID":"86e0029a-8eb5-4ce8-9151-6a47bf8111bb","Handler":null,"Name":"","Endpoint":""} Jul 09 09:54:46.265160 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.265132 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 09:54:46.265236 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.265181 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 09:54:46.324212 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.324190 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:46.324317 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.324216 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:46.324317 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:46.324294 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:46.324479 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:46.324455 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:46.455894 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.455869 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" event={"ID":"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4","Type":"ContainerStarted","Data":"0f4634ed99ebd7b5225283be95d18733788b211e0e9140036b27355244152adf"} Jul 09 09:54:46.486255 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:46.486200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:47.324475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.324296 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:47.324637 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:47.324551 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:47.341279 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.341248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:47.409714 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.409689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:47.460259 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.460229 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" event={"ID":"09fb5f7b-3db9-4f3c-94c4-bc698d3b8de4","Type":"ContainerStarted","Data":"4c06e312afd699a12b71728405ca764c323cb1c136fe3eeb4e07443ad8a608ce"} Jul 09 09:54:47.464036 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.464016 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"46b8beea1705e8356367f9900d6c2e7b825bfad95926b12e33784c4c2d628416"} Jul 09 09:54:47.482365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.482304 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-25hsp" podStartSLOduration=3.77317748 podStartE2EDuration="25.482284073s" totalImagesPullingTime="21.709106593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:47.481726657 +0000 UTC m=+25.679818457" watchObservedRunningTime="2026-07-09 09:54:47.482284073 +0000 UTC m=+25.680375871" Jul 09 09:54:47.482365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.482346 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:47.583226 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.583132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:47.783840 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.783803 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:47.784353 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:47.784334 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:48.324674 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.324640 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:48.324851 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.324644 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:48.324851 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:48.324789 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:48.324851 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:48.324835 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:48.466091 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.466059 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:48.466820 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.466677 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8prlt" Jul 09 09:54:48.547810 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.547779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:48.708171 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.708084 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wvb64_0115514f-42d4-40a8-94ea-cfb3069fa696/dns-node-resolver/0.log" Jul 09 09:54:48.711364 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:48.711338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:49.306021 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:49.305993 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvxlb_3014cfa9-cf7d-4b35-bd09-ff468ba689a2/node-ca/0.log" Jul 09 09:54:49.324314 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:49.324290 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:49.324427 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:49.324388 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:50.324367 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.324342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:50.324758 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:50.324445 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:50.324758 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.324521 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:50.324758 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:50.324617 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:50.474545 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.474387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" event={"ID":"d153a1ca-b6ce-4f28-b715-0b183167826b","Type":"ContainerStarted","Data":"57ae98762d178391a313451d0f718270047e295ed378116a7ff0fc46708de0a5"} Jul 09 09:54:50.474680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.474661 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:50.474760 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.474721 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:50.474813 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.474762 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:50.490854 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.490834 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:50.490972 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.490960 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:54:50.504279 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:50.504240 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" podStartSLOduration=9.689074041 podStartE2EDuration="28.504227184s" totalImagesPullingTime="18.815153143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:50.503557314 +0000 UTC m=+28.701649125" watchObservedRunningTime="2026-07-09 09:54:50.504227184 +0000 UTC m=+28.702318981" Jul 09 09:54:51.324823 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:51.324794 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:51.325329 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:51.324886 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:51.477863 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:51.477829 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8e8a28e-d99d-4e19-86be-f23e804f0830" containerID="85e7b32de07b29b21eac9ee6b34b65d55689030ff6d440aaf22cd131c88747ad" exitCode=0 Jul 09 09:54:51.478003 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:51.477911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerDied","Data":"85e7b32de07b29b21eac9ee6b34b65d55689030ff6d440aaf22cd131c88747ad"} Jul 09 09:54:52.326035 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.325964 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:52.326566 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:52.326079 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:52.326566 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.326418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:52.326566 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:52.326511 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:52.511298 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.511111 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:52.554679 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.554644 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hpl6x"] Jul 09 09:54:52.554811 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.554768 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:52.554891 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:52.554865 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:52.555510 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.555479 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xdxhj"] Jul 09 09:54:52.555985 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.555961 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:52.558805 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:52.556200 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:52.558805 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.557125 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tlvqc"] Jul 09 09:54:52.558805 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.557292 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:52.558805 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:52.557438 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:52.611043 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:52.610991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:53.537137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:53.537108 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:53.638076 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:53.638046 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:53.894091 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:53.894060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:53.999370 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:53.999342 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:54.078573 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.078523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:54.179542 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.179486 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:54.281993 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.281963 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:54.325051 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.325023 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:54.325193 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.325023 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:54.325193 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:54.325131 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:54.325302 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:54.325211 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:54.325302 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.325272 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:54.325398 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:54.325368 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:54.486801 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.486770 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8e8a28e-d99d-4e19-86be-f23e804f0830" containerID="34782c26797a959f45da4330bc84e68dbf7e96c377e05ce52f5b7275bb2eb91a" exitCode=0 Jul 09 09:54:54.486928 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:54.486821 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerDied","Data":"34782c26797a959f45da4330bc84e68dbf7e96c377e05ce52f5b7275bb2eb91a"} Jul 09 09:54:55.493516 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:55.493481 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8e8a28e-d99d-4e19-86be-f23e804f0830" containerID="9d817674a4f3a2aabe6344fa007703438ec09fba3a1a7d11b008e86b05a267e3" exitCode=0 Jul 09 09:54:55.494015 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:55.493529 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerDied","Data":"9d817674a4f3a2aabe6344fa007703438ec09fba3a1a7d11b008e86b05a267e3"} Jul 09 09:54:55.869863 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:55.869786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:55.932721 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:55.932688 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:55.932854 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:55.932816 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:55.932909 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:55.932878 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret podName:049c3668-cc57-448f-85bf-5f99de584f31 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:27.93286118 +0000 UTC m=+66.130952971 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret") pod "global-pull-secret-syncer-hpl6x" (UID: "049c3668-cc57-448f-85bf-5f99de584f31") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:55.934634 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:55.934612 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:56.033054 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:56.033032 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:56.033214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:56.033086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:56.033280 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.033231 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:56.033280 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.033245 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:56.033280 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.033259 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vt6cf for pod openshift-network-diagnostics/network-check-target-xdxhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:56.033419 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.033308 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf podName:e5aec67b-a25f-4ac1-a57d-8ec90a62da47 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:28.033294865 +0000 UTC m=+66.231386660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt6cf" (UniqueName: "kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf") pod "network-check-target-xdxhj" (UID: "e5aec67b-a25f-4ac1-a57d-8ec90a62da47") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:56.033419 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.033229 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:56.033419 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.033371 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs podName:85c9b5c9-46e7-4084-a466-a715bf7b6e2f nodeName:}" failed. No retries permitted until 2026-07-09 09:55:28.033353784 +0000 UTC m=+66.231445568 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs") pod "network-metrics-daemon-tlvqc" (UID: "85c9b5c9-46e7-4084-a466-a715bf7b6e2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:56.324450 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:56.324420 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:56.324617 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:56.324432 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:56.324617 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.324531 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xdxhj" podUID="e5aec67b-a25f-4ac1-a57d-8ec90a62da47" Jul 09 09:54:56.324617 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:56.324547 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:56.324745 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.324616 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tlvqc" podUID="85c9b5c9-46e7-4084-a466-a715bf7b6e2f" Jul 09 09:54:56.324745 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:54:56.324691 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpl6x" podUID="049c3668-cc57-448f-85bf-5f99de584f31" Jul 09 09:54:56.772304 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:56.772261 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:54:57.061210 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:57.061122 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeReady" Jul 09 09:54:57.061362 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:57.061273 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 09:54:58.324470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.324425 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:54:58.324470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.324462 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:54:58.324983 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.324486 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:54:58.329325 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.329188 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 09:54:58.329325 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.329202 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 09:54:58.329325 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.329207 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4794j\"" Jul 09 09:54:58.329325 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.329218 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ffmqr\"" Jul 09 09:54:58.329325 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.329242 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 09:54:58.329325 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:54:58.329244 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 09:55:00.165602 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.165374 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:00.461794 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.461734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:00.508398 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.508364 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8e8a28e-d99d-4e19-86be-f23e804f0830" containerID="42997e43c38e76f3779894f32659a700a2b6e4b871ee4d0f6480033fa987a78a" exitCode=0 Jul 09 09:55:00.508526 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.508426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerDied","Data":"42997e43c38e76f3779894f32659a700a2b6e4b871ee4d0f6480033fa987a78a"} Jul 09 09:55:00.585726 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.585692 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:00.685869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.685842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:00.786629 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.786605 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:00.887103 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.887071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:00.987920 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:00.987891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:01.514240 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:01.514211 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8e8a28e-d99d-4e19-86be-f23e804f0830" containerID="75c6a45d185eecf52cc2ba686796e5fec21eddb94f2d009f8d066aaee9d22d5f" exitCode=0 Jul 09 09:55:01.514592 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:01.514266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerDied","Data":"75c6a45d185eecf52cc2ba686796e5fec21eddb94f2d009f8d066aaee9d22d5f"} Jul 09 09:55:01.561937 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:01.561911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:01.806886 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:01.806806 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:01.868078 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:01.868048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:01.945053 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:01.945023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:02.156678 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:02.156605 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:02.519599 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:02.519565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" event={"ID":"e8e8a28e-d99d-4e19-86be-f23e804f0830","Type":"ContainerStarted","Data":"bcf75018727422f2574b675eb15b0996d0b9f64e82264028fdad710c5a9a327d"} Jul 09 09:55:02.546779 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:02.546737 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b2vq9" podStartSLOduration=8.677692448 podStartE2EDuration="40.546722814s" totalImagesPullingTime="31.869030366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:02.545209737 +0000 UTC m=+40.743301534" watchObservedRunningTime="2026-07-09 09:55:02.546722814 +0000 UTC m=+40.744814810" Jul 09 09:55:02.558369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:02.558338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:03.161085 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:03.161043 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:03.306337 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:03.306309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:03.549800 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:03.549772 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:03.946434 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:03.946334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:04.149019 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:04.148986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:04.343080 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:04.343049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:04.411944 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:04.411903 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:04.559253 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:04.559218 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:04.942768 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:04.942738 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:05.043236 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:05.043201 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:05.750009 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:05.749974 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:05.979293 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:05.979260 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:06.269192 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:06.269141 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:06.352120 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:06.352089 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:06.547907 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:06.547823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:07.147678 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:07.147646 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:07.759732 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:07.759689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:07.947248 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:07.947211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:08.351199 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:08.351161 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:08.544027 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:08.543997 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:08.952522 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:08.952478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:09.748494 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:09.748456 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:09.952861 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:09.952827 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:10.347113 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:10.347081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:10.549044 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:10.549011 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:10.943898 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:10.943869 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:11.159633 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:11.159596 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:15.836991 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.836951 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:15.933827 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.933789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:15.956379 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.956357 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x6k8c"] Jul 09 09:55:15.977767 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.977747 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gcvsp"] Jul 09 09:55:15.977873 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.977857 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:15.980543 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.980526 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n6pb2\"" Jul 09 09:55:15.980663 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.980527 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 09:55:15.980929 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.980913 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 09:55:15.999659 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.999635 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x6k8c"] Jul 09 09:55:15.999744 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.999673 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcvsp"] Jul 09 09:55:15.999744 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:15.999738 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.002074 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.002058 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 09:55:16.002182 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.002082 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-df4wp\"" Jul 09 09:55:16.002435 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.002418 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 09:55:16.002512 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.002464 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 09:55:16.003199 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.003178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:16.059744 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.059723 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d2ptr"] Jul 09 09:55:16.071277 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.071257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cff2144a-a21f-4900-9081-7078900fb02f-config-volume\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.071352 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.071283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5htkk\" (UniqueName: \"kubernetes.io/projected/cff2144a-a21f-4900-9081-7078900fb02f-kube-api-access-5htkk\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.071352 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.071304 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cff2144a-a21f-4900-9081-7078900fb02f-tmp-dir\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.071420 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.071394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6336c87-140b-4171-a441-2a99d62fbb1e-cert\") pod \"ingress-canary-gcvsp\" (UID: \"d6336c87-140b-4171-a441-2a99d62fbb1e\") " pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.071452 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.071428 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cff2144a-a21f-4900-9081-7078900fb02f-metrics-tls\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.071452 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.071447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m9lr\" (UniqueName: \"kubernetes.io/projected/d6336c87-140b-4171-a441-2a99d62fbb1e-kube-api-access-2m9lr\") pod \"ingress-canary-gcvsp\" (UID: \"d6336c87-140b-4171-a441-2a99d62fbb1e\") " pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.079790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.079771 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d2ptr"] Jul 09 09:55:16.079890 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.079856 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.082578 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.082559 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 09:55:16.082808 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.082789 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zkpxd\"" Jul 09 09:55:16.082882 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.082795 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 09:55:16.083078 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.083064 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 09:55:16.083123 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.083091 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 09:55:16.084813 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.084785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:16.169079 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.169019 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:16.171951 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.171936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cff2144a-a21f-4900-9081-7078900fb02f-tmp-dir\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.172012 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.171979 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-data-volume\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.172066 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6336c87-140b-4171-a441-2a99d62fbb1e-cert\") pod \"ingress-canary-gcvsp\" (UID: \"d6336c87-140b-4171-a441-2a99d62fbb1e\") " pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.172118 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-crio-socket\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.172118 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cff2144a-a21f-4900-9081-7078900fb02f-metrics-tls\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.172228 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m9lr\" (UniqueName: \"kubernetes.io/projected/d6336c87-140b-4171-a441-2a99d62fbb1e-kube-api-access-2m9lr\") pod \"ingress-canary-gcvsp\" (UID: \"d6336c87-140b-4171-a441-2a99d62fbb1e\") " pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.172228 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.172228 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172181 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm2k8\" (UniqueName: \"kubernetes.io/projected/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-kube-api-access-mm2k8\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.172228 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.172407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172246 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cff2144a-a21f-4900-9081-7078900fb02f-tmp-dir\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.172407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cff2144a-a21f-4900-9081-7078900fb02f-config-volume\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.172407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5htkk\" (UniqueName: \"kubernetes.io/projected/cff2144a-a21f-4900-9081-7078900fb02f-kube-api-access-5htkk\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.172939 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.172907 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cff2144a-a21f-4900-9081-7078900fb02f-config-volume\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.176021 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.176000 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cff2144a-a21f-4900-9081-7078900fb02f-metrics-tls\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.176097 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.176027 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6336c87-140b-4171-a441-2a99d62fbb1e-cert\") pod \"ingress-canary-gcvsp\" (UID: \"d6336c87-140b-4171-a441-2a99d62fbb1e\") " pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.185533 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.185427 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m9lr\" (UniqueName: \"kubernetes.io/projected/d6336c87-140b-4171-a441-2a99d62fbb1e-kube-api-access-2m9lr\") pod \"ingress-canary-gcvsp\" (UID: \"d6336c87-140b-4171-a441-2a99d62fbb1e\") " pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.189372 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.189345 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5htkk\" (UniqueName: \"kubernetes.io/projected/cff2144a-a21f-4900-9081-7078900fb02f-kube-api-access-5htkk\") pod \"dns-default-x6k8c\" (UID: \"cff2144a-a21f-4900-9081-7078900fb02f\") " pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.263722 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.263695 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:16.272819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.272796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-crio-socket\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.272890 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.272825 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.272890 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.272842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm2k8\" (UniqueName: \"kubernetes.io/projected/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-kube-api-access-mm2k8\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.272890 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.272859 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.272986 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.272960 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-crio-socket\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.273047 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.273020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-data-volume\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.273319 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.273300 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-data-volume\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.273407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.273391 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.274749 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.274734 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.281635 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.281618 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm2k8\" (UniqueName: \"kubernetes.io/projected/eead6a6b-baf5-44ec-a31c-a7aacb7a7423-kube-api-access-mm2k8\") pod \"insights-runtime-extractor-d2ptr\" (UID: \"eead6a6b-baf5-44ec-a31c-a7aacb7a7423\") " pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.286049 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.286032 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:16.308104 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.308081 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcvsp" Jul 09 09:55:16.388842 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.388812 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d2ptr" Jul 09 09:55:16.513466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.513440 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcvsp"] Jul 09 09:55:16.516266 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.516239 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x6k8c"] Jul 09 09:55:16.517026 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:16.517003 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6336c87_140b_4171_a441_2a99d62fbb1e.slice/crio-7abc49f6fcc50218fb9479c09e246c117d550466bba87c18d610b68291a96f70 WatchSource:0}: Error finding container 7abc49f6fcc50218fb9479c09e246c117d550466bba87c18d610b68291a96f70: Status 404 returned error can't find the container with id 7abc49f6fcc50218fb9479c09e246c117d550466bba87c18d610b68291a96f70 Jul 09 09:55:16.519337 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:16.519310 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff2144a_a21f_4900_9081_7078900fb02f.slice/crio-fbabf062daa43ba914a3e11eb45bf8436ae3b1dcefb400bb9588f2c62611eef6 WatchSource:0}: Error finding container fbabf062daa43ba914a3e11eb45bf8436ae3b1dcefb400bb9588f2c62611eef6: Status 404 returned error can't find the container with id fbabf062daa43ba914a3e11eb45bf8436ae3b1dcefb400bb9588f2c62611eef6 Jul 09 09:55:16.538733 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.538713 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d2ptr"] Jul 09 09:55:16.541361 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:16.541342 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeead6a6b_baf5_44ec_a31c_a7aacb7a7423.slice/crio-3913fba6f503850bf5d0d505ea5dabfaa27fb5d76a3f657569402bac5ea30fb4 WatchSource:0}: Error finding container 3913fba6f503850bf5d0d505ea5dabfaa27fb5d76a3f657569402bac5ea30fb4: Status 404 returned error can't find the container with id 3913fba6f503850bf5d0d505ea5dabfaa27fb5d76a3f657569402bac5ea30fb4 Jul 09 09:55:16.550515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.550490 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d2ptr" event={"ID":"eead6a6b-baf5-44ec-a31c-a7aacb7a7423","Type":"ContainerStarted","Data":"3913fba6f503850bf5d0d505ea5dabfaa27fb5d76a3f657569402bac5ea30fb4"} Jul 09 09:55:16.551458 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.551439 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcvsp" event={"ID":"d6336c87-140b-4171-a441-2a99d62fbb1e","Type":"ContainerStarted","Data":"7abc49f6fcc50218fb9479c09e246c117d550466bba87c18d610b68291a96f70"} Jul 09 09:55:16.552309 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:16.552293 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x6k8c" event={"ID":"cff2144a-a21f-4900-9081-7078900fb02f","Type":"ContainerStarted","Data":"fbabf062daa43ba914a3e11eb45bf8436ae3b1dcefb400bb9588f2c62611eef6"} Jul 09 09:55:18.416166 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.416051 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:18.498833 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.498763 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:18.561110 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.560640 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x6k8c" event={"ID":"cff2144a-a21f-4900-9081-7078900fb02f","Type":"ContainerStarted","Data":"60badd844d66f326c90a5f8297ed510fc55aff43ecf705ff309d3f7821321b7c"} Jul 09 09:55:18.561110 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.560704 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x6k8c" event={"ID":"cff2144a-a21f-4900-9081-7078900fb02f","Type":"ContainerStarted","Data":"e8c7de83e1bac6897779775172aa23b099ac10dea5be19bc375aaee3e74eb88e"} Jul 09 09:55:18.561110 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.560825 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:18.581802 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.581711 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:18.581991 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.581948 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x6k8c" podStartSLOduration=2.052573641 podStartE2EDuration="3.581930411s" totalImagesPullingTime="1.52935677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:18.581682066 +0000 UTC m=+56.779773864" watchObservedRunningTime="2026-07-09 09:55:18.581930411 +0000 UTC m=+56.780022209" Jul 09 09:55:18.603668 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.602830 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr"] Jul 09 09:55:18.604847 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.604826 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:18.608223 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.608011 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-74ckh\"" Jul 09 09:55:18.608223 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.608135 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 09:55:18.617392 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.617369 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr"] Jul 09 09:55:18.694723 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.694648 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/614004be-6dda-4486-8120-cfc0bfea0536-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d97xr\" (UID: \"614004be-6dda-4486-8120-cfc0bfea0536\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:18.742796 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.742765 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:18.795984 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:18.795947 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/614004be-6dda-4486-8120-cfc0bfea0536-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d97xr\" (UID: \"614004be-6dda-4486-8120-cfc0bfea0536\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:18.796130 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:18.796079 2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 09 09:55:18.796219 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:18.796137 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/614004be-6dda-4486-8120-cfc0bfea0536-tls-certificates podName:614004be-6dda-4486-8120-cfc0bfea0536 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:19.296122648 +0000 UTC m=+57.494214442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/614004be-6dda-4486-8120-cfc0bfea0536-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-d97xr" (UID: "614004be-6dda-4486-8120-cfc0bfea0536") : secret "prometheus-operator-admission-webhook-tls" not found Jul 09 09:55:19.008722 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:19.008690 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:19.300765 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:19.300689 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/614004be-6dda-4486-8120-cfc0bfea0536-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d97xr\" (UID: \"614004be-6dda-4486-8120-cfc0bfea0536\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:19.303322 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:19.303294 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/614004be-6dda-4486-8120-cfc0bfea0536-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d97xr\" (UID: \"614004be-6dda-4486-8120-cfc0bfea0536\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:19.517390 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:19.517353 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:22.282050 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.281979 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr"] Jul 09 09:55:22.285802 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:22.285778 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614004be_6dda_4486_8120_cfc0bfea0536.slice/crio-990eefddfe682ce8126f4df8acc3fd9a931582f91fe16b0cc0658f3c03ec10ca WatchSource:0}: Error finding container 990eefddfe682ce8126f4df8acc3fd9a931582f91fe16b0cc0658f3c03ec10ca: Status 404 returned error can't find the container with id 990eefddfe682ce8126f4df8acc3fd9a931582f91fe16b0cc0658f3c03ec10ca Jul 09 09:55:22.489938 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.489908 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l7cws" Jul 09 09:55:22.572340 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.572309 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d2ptr" event={"ID":"eead6a6b-baf5-44ec-a31c-a7aacb7a7423","Type":"ContainerStarted","Data":"4306cdce3d449ee3a088dbaf7c64d2f54b0efc9414a443d5e1f6222be50c3497"} Jul 09 09:55:22.573488 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.573461 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" event={"ID":"614004be-6dda-4486-8120-cfc0bfea0536","Type":"ContainerStarted","Data":"990eefddfe682ce8126f4df8acc3fd9a931582f91fe16b0cc0658f3c03ec10ca"} Jul 09 09:55:22.574705 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.574684 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcvsp" event={"ID":"d6336c87-140b-4171-a441-2a99d62fbb1e","Type":"ContainerStarted","Data":"94339f68a231ce9a929800a4a5f269049eb682916b785c05dbc9e475d9527bd2"} Jul 09 09:55:22.591387 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.591343 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gcvsp" podStartSLOduration=1.944523786 podStartE2EDuration="7.591329903s" totalImagesPullingTime="5.646806117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:22.590396994 +0000 UTC m=+60.788488792" watchObservedRunningTime="2026-07-09 09:55:22.591329903 +0000 UTC m=+60.789421699" Jul 09 09:55:22.624609 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.624578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:22.686105 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:22.686071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:23.578778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:23.578737 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d2ptr" event={"ID":"eead6a6b-baf5-44ec-a31c-a7aacb7a7423","Type":"ContainerStarted","Data":"d0fbf6797bbe2a5835add2b5cdac501d4ad8d9557c0316b9cf33d7102fa0968c"} Jul 09 09:55:24.584169 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:24.584125 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" event={"ID":"614004be-6dda-4486-8120-cfc0bfea0536","Type":"ContainerStarted","Data":"16294c358591e74c8824bedc230a79113958ce5611a170a3b6a1ce40fe064594"} Jul 09 09:55:24.584619 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:24.584373 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:24.589628 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:24.589605 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" Jul 09 09:55:24.602055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:24.602010 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d97xr" podStartSLOduration=5.008056364 podStartE2EDuration="6.601993341s" totalImagesPullingTime="1.593936977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:24.600913072 +0000 UTC m=+62.799004870" watchObservedRunningTime="2026-07-09 09:55:24.601993341 +0000 UTC m=+62.800085139" Jul 09 09:55:25.616019 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.615977 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:25.657196 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.657144 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj"] Jul 09 09:55:25.667219 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.667191 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.671312 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.671289 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 09:55:25.672554 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.672527 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 09:55:25.672554 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.672526 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m62rb\"" Jul 09 09:55:25.672716 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.672539 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 09:55:25.672716 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.672699 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 09:55:25.672819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.672763 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 09:55:25.673720 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.673698 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj"] Jul 09 09:55:25.678375 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.678305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:25.750010 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.749979 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.750137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.750050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4m9f\" (UniqueName: \"kubernetes.io/projected/7c3c684d-c416-467e-8229-c072540001f1-kube-api-access-x4m9f\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.750137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.750073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.750137 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.750095 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3c684d-c416-467e-8229-c072540001f1-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.851255 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.851235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.851345 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.851294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4m9f\" (UniqueName: \"kubernetes.io/projected/7c3c684d-c416-467e-8229-c072540001f1-kube-api-access-x4m9f\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.851345 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.851319 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.851416 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.851343 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3c684d-c416-467e-8229-c072540001f1-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.851416 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:25.851363 2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 09:55:25.851416 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:25.851410 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-tls podName:7c3c684d-c416-467e-8229-c072540001f1 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:26.351395521 +0000 UTC m=+64.549487296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-n6trj" (UID: "7c3c684d-c416-467e-8229-c072540001f1") : secret "prometheus-operator-tls" not found Jul 09 09:55:25.851934 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.851912 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3c684d-c416-467e-8229-c072540001f1-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.853295 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.853272 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:25.860040 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:25.860019 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4m9f\" (UniqueName: \"kubernetes.io/projected/7c3c684d-c416-467e-8229-c072540001f1-kube-api-access-x4m9f\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:26.180000 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.179960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:26.355473 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.355444 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:26.357726 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.357701 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c3c684d-c416-467e-8229-c072540001f1-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-n6trj\" (UID: \"7c3c684d-c416-467e-8229-c072540001f1\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:26.581281 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.581255 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" Jul 09 09:55:26.593096 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.593065 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d2ptr" event={"ID":"eead6a6b-baf5-44ec-a31c-a7aacb7a7423","Type":"ContainerStarted","Data":"f5f73d40f7e759bba87619a9f3d97d436888e0b4d57e8574bd0042e9e3966960"} Jul 09 09:55:26.616771 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.616375 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d2ptr" podStartSLOduration=1.514332246 podStartE2EDuration="10.616354526s" totalImagesPullingTime="9.10202228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:55:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:26.615194431 +0000 UTC m=+64.813286228" watchObservedRunningTime="2026-07-09 09:55:26.616354526 +0000 UTC m=+64.814446322" Jul 09 09:55:26.699852 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:26.699824 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj"] Jul 09 09:55:26.702735 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:26.702712 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3c684d_c416_467e_8229_c072540001f1.slice/crio-9818bf99d1a19167c0d87573515b675289233d8c7cf325bea8853be82ac99b93 WatchSource:0}: Error finding container 9818bf99d1a19167c0d87573515b675289233d8c7cf325bea8853be82ac99b93: Status 404 returned error can't find the container with id 9818bf99d1a19167c0d87573515b675289233d8c7cf325bea8853be82ac99b93 Jul 09 09:55:27.572338 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:27.572297 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:27.597625 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:27.597592 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" event={"ID":"7c3c684d-c416-467e-8229-c072540001f1","Type":"ContainerStarted","Data":"9818bf99d1a19167c0d87573515b675289233d8c7cf325bea8853be82ac99b93"} Jul 09 09:55:27.637004 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:27.636980 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:27.967995 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:27.967955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:55:27.971038 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:27.971017 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 09:55:27.981583 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:27.981557 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/049c3668-cc57-448f-85bf-5f99de584f31-original-pull-secret\") pod \"global-pull-secret-syncer-hpl6x\" (UID: \"049c3668-cc57-448f-85bf-5f99de584f31\") " pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:55:28.049961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.049929 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpl6x" Jul 09 09:55:28.069045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.069023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:55:28.069190 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.069074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:55:28.074447 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.074423 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 09:55:28.074447 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.074440 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 09:55:28.082160 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.081682 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c9b5c9-46e7-4084-a466-a715bf7b6e2f-metrics-certs\") pod \"network-metrics-daemon-tlvqc\" (UID: \"85c9b5c9-46e7-4084-a466-a715bf7b6e2f\") " pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:55:28.082580 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.082559 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 09:55:28.092729 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.092266 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6cf\" (UniqueName: \"kubernetes.io/projected/e5aec67b-a25f-4ac1-a57d-8ec90a62da47-kube-api-access-vt6cf\") pod \"network-check-target-xdxhj\" (UID: \"e5aec67b-a25f-4ac1-a57d-8ec90a62da47\") " pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:55:28.185194 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.185167 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hpl6x"] Jul 09 09:55:28.187629 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:28.187608 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049c3668_cc57_448f_85bf_5f99de584f31.slice/crio-e0d6ea01071778dce554340a57feb51988b81a949921b65bbc1ce2e1ac81bd37 WatchSource:0}: Error finding container e0d6ea01071778dce554340a57feb51988b81a949921b65bbc1ce2e1ac81bd37: Status 404 returned error can't find the container with id e0d6ea01071778dce554340a57feb51988b81a949921b65bbc1ce2e1ac81bd37 Jul 09 09:55:28.339166 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.339096 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ffmqr\"" Jul 09 09:55:28.346569 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.346547 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tlvqc" Jul 09 09:55:28.346892 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.346869 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4794j\"" Jul 09 09:55:28.355183 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.355117 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:55:28.566243 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.566216 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x6k8c" Jul 09 09:55:28.602395 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.602314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hpl6x" event={"ID":"049c3668-cc57-448f-85bf-5f99de584f31","Type":"ContainerStarted","Data":"e0d6ea01071778dce554340a57feb51988b81a949921b65bbc1ce2e1ac81bd37"} Jul 09 09:55:28.674078 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.674021 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:28.779103 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.778142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:28.788477 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.788388 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xdxhj"] Jul 09 09:55:28.801106 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:28.801063 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5aec67b_a25f_4ac1_a57d_8ec90a62da47.slice/crio-887d0c4b2ed53ca93604f231f26a332c4e1bc16fab31310b3d1f89792e202fd1 WatchSource:0}: Error finding container 887d0c4b2ed53ca93604f231f26a332c4e1bc16fab31310b3d1f89792e202fd1: Status 404 returned error can't find the container with id 887d0c4b2ed53ca93604f231f26a332c4e1bc16fab31310b3d1f89792e202fd1 Jul 09 09:55:28.810944 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.810916 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tlvqc"] Jul 09 09:55:28.813961 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:28.813916 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c9b5c9_46e7_4084_a466_a715bf7b6e2f.slice/crio-f0f6dc2a8c836a69424f014ff203b40170432dca9cd4522e6ebf31497ea7d701 WatchSource:0}: Error finding container f0f6dc2a8c836a69424f014ff203b40170432dca9cd4522e6ebf31497ea7d701: Status 404 returned error can't find the container with id f0f6dc2a8c836a69424f014ff203b40170432dca9cd4522e6ebf31497ea7d701 Jul 09 09:55:28.981347 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:28.981315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:29.608382 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:29.608326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" event={"ID":"7c3c684d-c416-467e-8229-c072540001f1","Type":"ContainerStarted","Data":"1c622e67adc69ef690e8b571d9f3721770e2a356afeaa4b3e5b51bd3cc9d2ac4"} Jul 09 09:55:29.608382 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:29.608378 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" event={"ID":"7c3c684d-c416-467e-8229-c072540001f1","Type":"ContainerStarted","Data":"70ed202ffbd3cce541d36753e624e401d9d7c8279d2564bd38d058d6f6d1147b"} Jul 09 09:55:29.610562 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:29.610481 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tlvqc" event={"ID":"85c9b5c9-46e7-4084-a466-a715bf7b6e2f","Type":"ContainerStarted","Data":"f0f6dc2a8c836a69424f014ff203b40170432dca9cd4522e6ebf31497ea7d701"} Jul 09 09:55:29.611926 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:29.611899 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xdxhj" event={"ID":"e5aec67b-a25f-4ac1-a57d-8ec90a62da47","Type":"ContainerStarted","Data":"887d0c4b2ed53ca93604f231f26a332c4e1bc16fab31310b3d1f89792e202fd1"} Jul 09 09:55:29.635095 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:29.635037 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-n6trj" podStartSLOduration=2.665145143 podStartE2EDuration="4.635018447s" totalImagesPullingTime="1.969873304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:29.629940418 +0000 UTC m=+67.828032215" watchObservedRunningTime="2026-07-09 09:55:29.635018447 +0000 UTC m=+67.833110229" Jul 09 09:55:30.618637 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:30.618550 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tlvqc" event={"ID":"85c9b5c9-46e7-4084-a466-a715bf7b6e2f","Type":"ContainerStarted","Data":"599934c20f8ffdcf4c4ea6d8916944b51239a8b7768fc559271d38e0580516aa"} Jul 09 09:55:30.618637 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:30.618611 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tlvqc" event={"ID":"85c9b5c9-46e7-4084-a466-a715bf7b6e2f","Type":"ContainerStarted","Data":"b50434954b03e781a5c0ba0e769c1887c3468554cfea3850f1f7a7c8d97c49ec"} Jul 09 09:55:30.638128 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:30.638078 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tlvqc" podStartSLOduration=67.449085303 podStartE2EDuration="1m8.638057701s" totalImagesPullingTime="1.188972398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:30.636897868 +0000 UTC m=+68.834989667" watchObservedRunningTime="2026-07-09 09:55:30.638057701 +0000 UTC m=+68.836149517" Jul 09 09:55:31.355209 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:31.355172 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:31.455673 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:31.455631 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:31.786636 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:31.786552 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:31.940288 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:31.940253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.042854 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.042780 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr"] Jul 09 09:55:32.045046 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.045022 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.047748 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.047725 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 09:55:32.049214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.049025 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 09:55:32.049214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.049114 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 09:55:32.049214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.049127 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4tq9g\"" Jul 09 09:55:32.050713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.050690 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2"] Jul 09 09:55:32.052964 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.052938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.053665 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.053637 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.056235 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.056111 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 09:55:32.056235 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.056144 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 09:55:32.056401 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.056268 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f5hh6\"" Jul 09 09:55:32.057916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.057871 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr"] Jul 09 09:55:32.064884 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.064863 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2"] Jul 09 09:55:32.074897 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.074879 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-brsxs"] Jul 09 09:55:32.076997 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.076976 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.079786 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.079757 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 09:55:32.079881 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.079820 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ffn49\"" Jul 09 09:55:32.079881 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.079872 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 09:55:32.079984 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.079763 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 09:55:32.101360 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2bd1c6f-60e5-435f-952f-0e15f618b343-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.101459 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvfvh\" (UniqueName: \"kubernetes.io/projected/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-api-access-dvfvh\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.101459 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101407 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.101459 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kkvq\" (UniqueName: \"kubernetes.io/projected/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-kube-api-access-6kkvq\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.101459 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101457 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f2bd1c6f-60e5-435f-952f-0e15f618b343-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.101649 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.101649 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.101649 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101599 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.101649 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.101811 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.101658 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.131891 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.131814 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.199362 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.199331 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.202283 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202256 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvfvh\" (UniqueName: \"kubernetes.io/projected/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-api-access-dvfvh\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.202428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-accelerators-collector-config\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202332 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.202428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202631 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-sys\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202631 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202531 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-tls\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202631 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvks8\" (UniqueName: \"kubernetes.io/projected/f927ac61-9c40-49b4-860a-7b44a3f16226-kube-api-access-xvks8\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202631 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f2bd1c6f-60e5-435f-952f-0e15f618b343-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202682 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f927ac61-9c40-49b4-860a-7b44a3f16226-metrics-client-ca\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:32.202692 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:32.202753 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-tls podName:e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d nodeName:}" failed. No retries permitted until 2026-07-09 09:55:32.702734555 +0000 UTC m=+70.900826332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-q9tv2" (UID: "e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d") : secret "openshift-state-metrics-tls" not found Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202788 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-wtmp\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:32.202810 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 09:55:32.202839 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202821 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:32.202859 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-tls podName:f2bd1c6f-60e5-435f-952f-0e15f618b343 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:32.702843524 +0000 UTC m=+70.900935317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-8v4wr" (UID: "f2bd1c6f-60e5-435f-952f-0e15f618b343") : secret "kube-state-metrics-tls" not found Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202879 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2bd1c6f-60e5-435f-952f-0e15f618b343-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kkvq\" (UniqueName: \"kubernetes.io/projected/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-kube-api-access-6kkvq\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.202982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.203010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-root\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.203068 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-textfile\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.203272 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.203083 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f2bd1c6f-60e5-435f-952f-0e15f618b343-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.203741 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.203693 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2bd1c6f-60e5-435f-952f-0e15f618b343-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.203821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.203770 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.204011 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.203987 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.205531 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.205510 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.205531 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.205527 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.214083 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.214059 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvfvh\" (UniqueName: \"kubernetes.io/projected/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-api-access-dvfvh\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.214352 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.214332 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kkvq\" (UniqueName: \"kubernetes.io/projected/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-kube-api-access-6kkvq\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.304188 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-root\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304188 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-textfile\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304191 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-accelerators-collector-config\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304222 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-sys\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304262 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-tls\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304272 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-root\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304280 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvks8\" (UniqueName: \"kubernetes.io/projected/f927ac61-9c40-49b4-860a-7b44a3f16226-kube-api-access-xvks8\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304384 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f927ac61-9c40-49b4-860a-7b44a3f16226-metrics-client-ca\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304724 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-wtmp\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304724 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304524 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-sys\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304724 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:32.304556 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 09:55:32.304724 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:32.304613 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-tls podName:f927ac61-9c40-49b4-860a-7b44a3f16226 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:32.804594447 +0000 UTC m=+71.002686225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-tls") pod "node-exporter-brsxs" (UID: "f927ac61-9c40-49b4-860a-7b44a3f16226") : secret "node-exporter-tls" not found Jul 09 09:55:32.304724 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304651 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-wtmp\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.304957 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.304795 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-textfile\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.305059 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.305039 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f927ac61-9c40-49b4-860a-7b44a3f16226-metrics-client-ca\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.305122 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.305091 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-accelerators-collector-config\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.307200 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.307137 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.311305 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.311215 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.315368 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.315343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvks8\" (UniqueName: \"kubernetes.io/projected/f927ac61-9c40-49b4-860a-7b44a3f16226-kube-api-access-xvks8\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.422770 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.422735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.524365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.524261 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.624434 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.624344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.709492 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.709451 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.709658 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.709513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.712439 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.712411 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2bd1c6f-60e5-435f-952f-0e15f618b343-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8v4wr\" (UID: \"f2bd1c6f-60e5-435f-952f-0e15f618b343\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.713068 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.713022 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-q9tv2\" (UID: \"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.731255 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.731218 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:32.810819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.810787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-tls\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.813135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.813108 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f927ac61-9c40-49b4-860a-7b44a3f16226-node-exporter-tls\") pod \"node-exporter-brsxs\" (UID: \"f927ac61-9c40-49b4-860a-7b44a3f16226\") " pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.955680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.955591 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" Jul 09 09:55:32.965831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.965802 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" Jul 09 09:55:32.986084 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.986059 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-brsxs" Jul 09 09:55:32.987863 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:32.987839 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:33.090331 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.090287 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:33.127328 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.127300 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:55:33.130024 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.130004 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.132784 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.132760 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.132800 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.132870 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133046 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133059 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133115 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mlzxw\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133132 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133064 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133119 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 09:55:33.133393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.133322 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 09:55:33.149186 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.149164 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:55:33.178378 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.178348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:33.214252 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214252 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214220 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214253 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214335 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214358 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clncf\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-kube-api-access-clncf\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214383 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-out\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214474 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214500 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214624 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.214671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.214666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-web-config\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315509 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315509 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315526 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315766 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315562 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315766 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315878 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315878 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315857 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-web-config\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315977 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315977 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.315977 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:33.315950 2576 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.315949 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:33.316368 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls podName:bd8a2cbd-ac13-4562-8f02-d9189f427578 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:33.815992236 +0000 UTC m=+72.014084031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578") : secret "alertmanager-main-tls" not found Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:55:33.316547 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle podName:bd8a2cbd-ac13-4562-8f02-d9189f427578 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:33.816527864 +0000 UTC m=+72.014619651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578") : configmap references non-existent config key: ca-bundle.crt Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.316580 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.316624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.316652 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clncf\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-kube-api-access-clncf\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.317062 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.317122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-out\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.317470 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.317242 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.320075 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.320046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.320357 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.320330 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.320460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.320436 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.322413 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.322332 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.322413 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.322367 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.323511 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.323475 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-web-config\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.323709 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.323662 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-out\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.323832 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.323769 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.327463 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.327311 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clncf\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-kube-api-access-clncf\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.416931 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.416889 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:33.472739 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.472546 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2"] Jul 09 09:55:33.475286 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:33.475259 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e53bdc_ec52_4e69_8e34_1f40b53c6e9d.slice/crio-057f600c77b0d568abe3c8181883ab71e69221181b3fb579083a4c781d432880 WatchSource:0}: Error finding container 057f600c77b0d568abe3c8181883ab71e69221181b3fb579083a4c781d432880: Status 404 returned error can't find the container with id 057f600c77b0d568abe3c8181883ab71e69221181b3fb579083a4c781d432880 Jul 09 09:55:33.491955 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.490813 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr"] Jul 09 09:55:33.500665 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:33.500631 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bd1c6f_60e5_435f_952f_0e15f618b343.slice/crio-fbd52488f357935844b2b4001b9dfd252dbffbab2979a82a5c92aa03598caac0 WatchSource:0}: Error finding container fbd52488f357935844b2b4001b9dfd252dbffbab2979a82a5c92aa03598caac0: Status 404 returned error can't find the container with id fbd52488f357935844b2b4001b9dfd252dbffbab2979a82a5c92aa03598caac0 Jul 09 09:55:33.630139 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.630107 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hpl6x" event={"ID":"049c3668-cc57-448f-85bf-5f99de584f31","Type":"ContainerStarted","Data":"6229485f2f9c8594fbc11457e3cc273b0c6818e34f2ca2c2a40ebe6436d85208"} Jul 09 09:55:33.631865 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.631802 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xdxhj" event={"ID":"e5aec67b-a25f-4ac1-a57d-8ec90a62da47","Type":"ContainerStarted","Data":"50c8354b6b93b3cd2c8d3b3286aece3bb169bb8b01c336f7af4be33d144136df"} Jul 09 09:55:33.631988 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.631925 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:55:33.633278 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.633250 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" event={"ID":"f2bd1c6f-60e5-435f-952f-0e15f618b343","Type":"ContainerStarted","Data":"fbd52488f357935844b2b4001b9dfd252dbffbab2979a82a5c92aa03598caac0"} Jul 09 09:55:33.635166 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.635136 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" event={"ID":"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d","Type":"ContainerStarted","Data":"a48f5cde43642e58dbb487d2e0bd3e069c624e18aba3b97dbe92847d2856053e"} Jul 09 09:55:33.635271 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.635197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" event={"ID":"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d","Type":"ContainerStarted","Data":"d2ba3eb9b2689e51c81b30bd00ec1e4aaa157cc6d416d2c05d35856fa8bcf0d3"} Jul 09 09:55:33.635271 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.635217 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" event={"ID":"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d","Type":"ContainerStarted","Data":"057f600c77b0d568abe3c8181883ab71e69221181b3fb579083a4c781d432880"} Jul 09 09:55:33.636236 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.636215 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-brsxs" event={"ID":"f927ac61-9c40-49b4-860a-7b44a3f16226","Type":"ContainerStarted","Data":"67530f697fa4a45c54e427e6a1fb37c44fb80f1907d1f7043ab9641fcc1ec9be"} Jul 09 09:55:33.648140 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.648095 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hpl6x" podStartSLOduration=66.510053144 podStartE2EDuration="1m11.648078831s" totalImagesPullingTime="5.138025687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:33.647387725 +0000 UTC m=+71.845479523" watchObservedRunningTime="2026-07-09 09:55:33.648078831 +0000 UTC m=+71.846170629" Jul 09 09:55:33.663828 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.663779 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xdxhj" podStartSLOduration=67.133746126 podStartE2EDuration="1m11.663763101s" totalImagesPullingTime="4.530016975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:33.662893788 +0000 UTC m=+71.860985586" watchObservedRunningTime="2026-07-09 09:55:33.663763101 +0000 UTC m=+71.861854902" Jul 09 09:55:33.823123 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.823012 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.823123 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.823099 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.824131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.824080 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.825878 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.825848 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:33.979369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:33.979334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:34.041304 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:34.041262 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:55:34.080502 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:34.080440 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:34.197700 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:34.197661 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:55:34.198266 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:34.198237 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8a2cbd_ac13_4562_8f02_d9189f427578.slice/crio-5685745a13b14d7f497ba9448e070d2432a020418e6a0ee260deb7f1672c253a WatchSource:0}: Error finding container 5685745a13b14d7f497ba9448e070d2432a020418e6a0ee260deb7f1672c253a: Status 404 returned error can't find the container with id 5685745a13b14d7f497ba9448e070d2432a020418e6a0ee260deb7f1672c253a Jul 09 09:55:34.643853 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:34.643813 2576 generic.go:356] "Generic (PLEG): container finished" podID="f927ac61-9c40-49b4-860a-7b44a3f16226" containerID="791f63acd4400731f44826af94ccec1cca771ed1dccbebdc18741156f6a5e73d" exitCode=0 Jul 09 09:55:34.644030 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:34.643918 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-brsxs" event={"ID":"f927ac61-9c40-49b4-860a-7b44a3f16226","Type":"ContainerDied","Data":"791f63acd4400731f44826af94ccec1cca771ed1dccbebdc18741156f6a5e73d"} Jul 09 09:55:34.646102 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:34.646069 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"5685745a13b14d7f497ba9448e070d2432a020418e6a0ee260deb7f1672c253a"} Jul 09 09:55:35.050919 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.050885 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:35.051514 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.051492 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j"] Jul 09 09:55:35.055387 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.055369 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.058033 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058005 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1lf7t7j9saiov\"" Jul 09 09:55:35.058138 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058033 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 09:55:35.058138 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058062 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 09:55:35.058380 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058146 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 09:55:35.058482 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058465 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 09:55:35.058698 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058683 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 09:55:35.058829 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.058711 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-msrpj\"" Jul 09 09:55:35.066623 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.066596 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j"] Jul 09 09:55:35.136230 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136203 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-tls\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136334 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136334 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc00daab-1825-4791-9274-ad4727a6640c-metrics-client-ca\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136334 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136490 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136490 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136381 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wf7l\" (UniqueName: \"kubernetes.io/projected/dc00daab-1825-4791-9274-ad4727a6640c-kube-api-access-5wf7l\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136490 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-grpc-tls\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.136490 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.136466 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.156634 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.156607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.237925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.237968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wf7l\" (UniqueName: \"kubernetes.io/projected/dc00daab-1825-4791-9274-ad4727a6640c-kube-api-access-5wf7l\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.238023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-grpc-tls\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.238062 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.238100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-tls\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.238139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.238190 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc00daab-1825-4791-9274-ad4727a6640c-metrics-client-ca\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.238427 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.238226 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.240295 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.239737 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc00daab-1825-4791-9274-ad4727a6640c-metrics-client-ca\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.242533 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.242476 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.243066 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.243038 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.243424 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.243401 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-tls\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.243574 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.243554 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.244317 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.244294 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.245361 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.245318 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc00daab-1825-4791-9274-ad4727a6640c-secret-grpc-tls\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.250057 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.250004 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wf7l\" (UniqueName: \"kubernetes.io/projected/dc00daab-1825-4791-9274-ad4727a6640c-kube-api-access-5wf7l\") pod \"thanos-querier-d8cddbb6f-4rm6j\" (UID: \"dc00daab-1825-4791-9274-ad4727a6640c\") " pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.259307 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.259251 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:35.364412 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.364384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:35.364549 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.364444 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:35.524736 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.524702 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j"] Jul 09 09:55:35.528858 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:35.528833 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc00daab_1825_4791_9274_ad4727a6640c.slice/crio-173e050bd5e92a5fb9e5855084baff33903ff6133332367a55536d756a2da0a9 WatchSource:0}: Error finding container 173e050bd5e92a5fb9e5855084baff33903ff6133332367a55536d756a2da0a9: Status 404 returned error can't find the container with id 173e050bd5e92a5fb9e5855084baff33903ff6133332367a55536d756a2da0a9 Jul 09 09:55:35.652368 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.652277 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" event={"ID":"f2bd1c6f-60e5-435f-952f-0e15f618b343","Type":"ContainerStarted","Data":"72b725c6ef4c055f840bdaeefc209227791860cfbc40c36a21c8dada0efbf28b"} Jul 09 09:55:35.652368 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.652346 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" event={"ID":"f2bd1c6f-60e5-435f-952f-0e15f618b343","Type":"ContainerStarted","Data":"2a801b3e143df80c7f8faf58cfcc019b91f256f4be0b0ae7b23517e677d9c105"} Jul 09 09:55:35.652566 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.652375 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" event={"ID":"f2bd1c6f-60e5-435f-952f-0e15f618b343","Type":"ContainerStarted","Data":"45bdd99610d0d9ba4d273a75f8d51c8be5202905a1c3cdc13d274aacab29600a"} Jul 09 09:55:35.653933 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.653902 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"173e050bd5e92a5fb9e5855084baff33903ff6133332367a55536d756a2da0a9"} Jul 09 09:55:35.656398 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.656368 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" event={"ID":"e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d","Type":"ContainerStarted","Data":"eceb6e6228ad0171b26f6ebcd12168202a66c3c2000ab91a536502a3c6483dbd"} Jul 09 09:55:35.658984 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.658965 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-brsxs" event={"ID":"f927ac61-9c40-49b4-860a-7b44a3f16226","Type":"ContainerStarted","Data":"f06c35aa4956ba24270fe88a40d9b577f73d2acb8efd22b3231a4e4f12b4931b"} Jul 09 09:55:35.659084 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.659007 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-brsxs" event={"ID":"f927ac61-9c40-49b4-860a-7b44a3f16226","Type":"ContainerStarted","Data":"b9e9e42d3a94232a9d50bc2adad5ba54f82c743faf67eb4a4bc2e864d96d18c3"} Jul 09 09:55:35.672930 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.672887 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8v4wr" podStartSLOduration=2.079910273 podStartE2EDuration="3.672869693s" totalImagesPullingTime="1.59295942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:35.671558002 +0000 UTC m=+73.869649794" watchObservedRunningTime="2026-07-09 09:55:35.672869693 +0000 UTC m=+73.870961496" Jul 09 09:55:35.691852 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.691799 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-brsxs" podStartSLOduration=2.842538927 podStartE2EDuration="3.691780113s" totalImagesPullingTime="785.404139ms" totalInitContainerRuntime="63.837047ms" isStatefulPod=true podCreationTimestamp="2026-07-09 09:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:35.689997655 +0000 UTC m=+73.888089477" watchObservedRunningTime="2026-07-09 09:55:35.691780113 +0000 UTC m=+73.889871912" Jul 09 09:55:35.710833 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:35.710623 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-q9tv2" podStartSLOduration=2.21958523 podStartE2EDuration="3.710605181s" totalImagesPullingTime="1.491019951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:35.709112202 +0000 UTC m=+73.907204002" watchObservedRunningTime="2026-07-09 09:55:35.710605181 +0000 UTC m=+73.908696978" Jul 09 09:55:36.357634 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.357554 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7bc595d68f-jvnrv"] Jul 09 09:55:36.360985 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.360960 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.365441 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.365074 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 09:55:36.365441 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.365078 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f58zc\"" Jul 09 09:55:36.365441 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.365352 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 09:55:36.365441 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.365128 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 09:55:36.365742 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.365196 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 09:55:36.365742 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.365200 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bpt0e1vqkj4cg\"" Jul 09 09:55:36.371819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.371797 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bc595d68f-jvnrv"] Jul 09 09:55:36.447884 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.447859 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.448042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.447907 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-secret-metrics-server-tls\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.448042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.447978 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdd2c\" (UniqueName: \"kubernetes.io/projected/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-kube-api-access-xdd2c\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.448042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.448022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-client-ca-bundle\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.448182 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.448048 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-metrics-server-audit-profiles\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.448182 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.448136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-secret-metrics-server-client-certs\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.448263 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.448179 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-audit-log\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549221 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549187 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-secret-metrics-server-tls\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549296 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdd2c\" (UniqueName: \"kubernetes.io/projected/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-kube-api-access-xdd2c\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-client-ca-bundle\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-metrics-server-audit-profiles\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549565 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-secret-metrics-server-client-certs\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549565 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549449 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-audit-log\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.549790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549768 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-audit-log\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.550004 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.549982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.550543 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.550509 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-metrics-server-audit-profiles\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.551651 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.551627 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-secret-metrics-server-tls\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.551815 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.551798 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-client-ca-bundle\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.551921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.551905 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-secret-metrics-server-client-certs\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.558420 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.558398 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdd2c\" (UniqueName: \"kubernetes.io/projected/a6d6fbd1-3bac-402a-91b9-45ddeaca2257-kube-api-access-xdd2c\") pod \"metrics-server-7bc595d68f-jvnrv\" (UID: \"a6d6fbd1-3bac-402a-91b9-45ddeaca2257\") " pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.665897 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.665806 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c" exitCode=0 Jul 09 09:55:36.666035 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.665909 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c"} Jul 09 09:55:36.674069 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.674049 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:36.811525 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.811443 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bc595d68f-jvnrv"] Jul 09 09:55:36.813782 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:36.813755 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d6fbd1_3bac_402a_91b9_45ddeaca2257.slice/crio-dd2c0b9ab9a764450226c1a5b0f281d8a2ae81f0e60a85fea306e2e3634bba45 WatchSource:0}: Error finding container dd2c0b9ab9a764450226c1a5b0f281d8a2ae81f0e60a85fea306e2e3634bba45: Status 404 returned error can't find the container with id dd2c0b9ab9a764450226c1a5b0f281d8a2ae81f0e60a85fea306e2e3634bba45 Jul 09 09:55:36.827135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.827114 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4"] Jul 09 09:55:36.829182 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.829137 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:36.831839 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.831820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:36.836369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.836181 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 09:55:36.836369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.836189 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gd25p\"" Jul 09 09:55:36.843367 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.843348 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4"] Jul 09 09:55:36.868518 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.868496 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-hxqvh"] Jul 09 09:55:36.871673 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.871650 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:55:36.874916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.874569 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 09:55:36.874916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.874788 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9cqsk\"" Jul 09 09:55:36.874916 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.874888 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 09:55:36.881369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.881348 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-hxqvh"] Jul 09 09:55:36.903358 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.903328 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:36.954989 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.954921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxw7q\" (UniqueName: \"kubernetes.io/projected/4e7b2884-a6b1-48ab-a959-e360e4fa153a-kube-api-access-sxw7q\") pod \"downloads-685bfb5f64-hxqvh\" (UID: \"4e7b2884-a6b1-48ab-a959-e360e4fa153a\") " pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:55:36.954989 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.954953 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5vgf\" (UniqueName: \"kubernetes.io/projected/0e025dbb-4bca-492f-a806-03b1575516bb-kube-api-access-h5vgf\") pod \"monitoring-plugin-d5dcff7cf-78dh4\" (UID: \"0e025dbb-4bca-492f-a806-03b1575516bb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:36.954989 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.954977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e025dbb-4bca-492f-a806-03b1575516bb-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-78dh4\" (UID: \"0e025dbb-4bca-492f-a806-03b1575516bb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:36.973814 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:36.973782 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:37.056345 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.056314 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxw7q\" (UniqueName: \"kubernetes.io/projected/4e7b2884-a6b1-48ab-a959-e360e4fa153a-kube-api-access-sxw7q\") pod \"downloads-685bfb5f64-hxqvh\" (UID: \"4e7b2884-a6b1-48ab-a959-e360e4fa153a\") " pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:55:37.056345 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.056347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5vgf\" (UniqueName: \"kubernetes.io/projected/0e025dbb-4bca-492f-a806-03b1575516bb-kube-api-access-h5vgf\") pod \"monitoring-plugin-d5dcff7cf-78dh4\" (UID: \"0e025dbb-4bca-492f-a806-03b1575516bb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:37.056525 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.056372 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e025dbb-4bca-492f-a806-03b1575516bb-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-78dh4\" (UID: \"0e025dbb-4bca-492f-a806-03b1575516bb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:37.058669 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.058649 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e025dbb-4bca-492f-a806-03b1575516bb-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-78dh4\" (UID: \"0e025dbb-4bca-492f-a806-03b1575516bb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:37.069803 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.069775 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxw7q\" (UniqueName: \"kubernetes.io/projected/4e7b2884-a6b1-48ab-a959-e360e4fa153a-kube-api-access-sxw7q\") pod \"downloads-685bfb5f64-hxqvh\" (UID: \"4e7b2884-a6b1-48ab-a959-e360e4fa153a\") " pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:55:37.071193 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.071175 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5vgf\" (UniqueName: \"kubernetes.io/projected/0e025dbb-4bca-492f-a806-03b1575516bb-kube-api-access-h5vgf\") pod \"monitoring-plugin-d5dcff7cf-78dh4\" (UID: \"0e025dbb-4bca-492f-a806-03b1575516bb\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:37.146446 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.146416 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:37.183048 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.183013 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:55:37.282574 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.282505 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7466fd7d65-fzfwp"] Jul 09 09:55:37.285869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.285847 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.288818 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.288785 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 09:55:37.288977 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.288946 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4"] Jul 09 09:55:37.289096 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.289025 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 09:55:37.289408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.289363 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 09:55:37.289408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.289369 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bkk94\"" Jul 09 09:55:37.289877 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.289799 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 09:55:37.290180 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.290162 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 09:55:37.298559 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.298527 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 09:55:37.309015 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.308995 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7466fd7d65-fzfwp"] Jul 09 09:55:37.358590 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h5t4\" (UniqueName: \"kubernetes.io/projected/6af3cdce-e40c-4843-af80-cff1d2939ef1-kube-api-access-7h5t4\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358644 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-telemeter-client-tls\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-serving-certs-ca-bundle\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358846 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-metrics-client-ca\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358884 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-federate-client-tls\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.358961 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.358943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-secret-telemeter-client\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.459718 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-secret-telemeter-client\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.459831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.459831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h5t4\" (UniqueName: \"kubernetes.io/projected/6af3cdce-e40c-4843-af80-cff1d2939ef1-kube-api-access-7h5t4\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.459831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459806 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-telemeter-client-tls\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.460003 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459850 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.460003 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-serving-certs-ca-bundle\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.460003 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-metrics-client-ca\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.460003 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.459986 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-federate-client-tls\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.460845 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.460813 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-serving-certs-ca-bundle\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.461131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.461074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-metrics-client-ca\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.461131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.461100 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af3cdce-e40c-4843-af80-cff1d2939ef1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.462882 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.462858 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-federate-client-tls\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.463329 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.463289 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-secret-telemeter-client\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.463884 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.463761 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.463884 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.463847 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6af3cdce-e40c-4843-af80-cff1d2939ef1-telemeter-client-tls\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.470193 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.470170 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h5t4\" (UniqueName: \"kubernetes.io/projected/6af3cdce-e40c-4843-af80-cff1d2939ef1-kube-api-access-7h5t4\") pod \"telemeter-client-7466fd7d65-fzfwp\" (UID: \"6af3cdce-e40c-4843-af80-cff1d2939ef1\") " pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.597922 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.597857 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" Jul 09 09:55:37.631377 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.631346 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:37.671870 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.671835 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" event={"ID":"a6d6fbd1-3bac-402a-91b9-45ddeaca2257","Type":"ContainerStarted","Data":"dd2c0b9ab9a764450226c1a5b0f281d8a2ae81f0e60a85fea306e2e3634bba45"} Jul 09 09:55:37.692190 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.692142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:37.754827 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:37.754784 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e025dbb_4bca_492f_a806_03b1575516bb.slice/crio-8b45b14887a83092879dc03ce131c6e655e358dc1c464d8fec43b9d8a4525404 WatchSource:0}: Error finding container 8b45b14887a83092879dc03ce131c6e655e358dc1c464d8fec43b9d8a4525404: Status 404 returned error can't find the container with id 8b45b14887a83092879dc03ce131c6e655e358dc1c464d8fec43b9d8a4525404 Jul 09 09:55:37.761263 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.761239 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:37.903624 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.903441 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-hxqvh"] Jul 09 09:55:37.906364 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:37.906336 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7b2884_a6b1_48ab_a959_e360e4fa153a.slice/crio-9aee71a827a53cd44383db53889ee70dd0eafbaa68a69dcbec18328ae05b0017 WatchSource:0}: Error finding container 9aee71a827a53cd44383db53889ee70dd0eafbaa68a69dcbec18328ae05b0017: Status 404 returned error can't find the container with id 9aee71a827a53cd44383db53889ee70dd0eafbaa68a69dcbec18328ae05b0017 Jul 09 09:55:37.931847 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:37.931825 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7466fd7d65-fzfwp"] Jul 09 09:55:37.933400 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:37.933376 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af3cdce_e40c_4843_af80_cff1d2939ef1.slice/crio-5aa15f1acab04c5b4d458531a140b9249145c38aded61a9f5f1df30eed03c34a WatchSource:0}: Error finding container 5aa15f1acab04c5b4d458531a140b9249145c38aded61a9f5f1df30eed03c34a: Status 404 returned error can't find the container with id 5aa15f1acab04c5b4d458531a140b9249145c38aded61a9f5f1df30eed03c34a Jul 09 09:55:38.333994 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.333911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:38.339447 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.339368 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:55:38.344671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.344646 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350279 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350389 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350498 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350610 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v8jtl\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350704 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350816 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350902 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.351093 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.351132 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.351306 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 09:55:38.351460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.351388 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 09:55:38.353376 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.350279 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 09:55:38.353376 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.351817 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-udh7e9hts1am\"" Jul 09 09:55:38.353750 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.351822 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 09:55:38.353750 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.352850 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 09:55:38.356069 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.356024 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 09:55:38.362173 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.361435 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:55:38.411525 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.411228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:38.470138 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.469991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470138 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470079 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470187 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-web-config\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470409 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470446 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470482 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470623 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x7bp\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-kube-api-access-7x7bp\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.470666 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.471093 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470694 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config-out\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.471093 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470774 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.471093 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.471093 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.470832 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.571812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.571870 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.571906 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.571943 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.571971 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.572001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.572025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.572056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.572081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x7bp\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-kube-api-access-7x7bp\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.572109 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.572622 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.572135 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config-out\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.575203 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.574819 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575519 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575568 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575595 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575640 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-web-config\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.575819 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.576511 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577615 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.577320 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577615 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.577436 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577615 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.577527 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577762 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.577684 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.577762 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.577755 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.578302 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.577974 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.578970 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.578838 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.579116 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.579094 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.580164 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.580127 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config-out\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.583042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.582605 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.583042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.582703 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.583042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.582722 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.583042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.582729 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.583042 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.583005 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.583381 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.583122 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-web-config\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.584455 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.584385 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.584999 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.584977 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.587819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.587793 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x7bp\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-kube-api-access-7x7bp\") pod \"prometheus-k8s-0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.664603 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.664381 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:38.679219 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.679015 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"b0eb150545ca2b922603e43819b843c609dfaa771138f1ceecb671df8d5dc7ce"} Jul 09 09:55:38.679219 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.679097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"3a5d86e26864f47234331335c8558bdb6db0804b1f939d2bafe267ce8b3e19fe"} Jul 09 09:55:38.679219 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.679137 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"53d25dfef1c43fa9875fec832fd9b1fb500b71da648728b378f532a238dcc8b8"} Jul 09 09:55:38.682276 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.682189 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" event={"ID":"6af3cdce-e40c-4843-af80-cff1d2939ef1","Type":"ContainerStarted","Data":"5aa15f1acab04c5b4d458531a140b9249145c38aded61a9f5f1df30eed03c34a"} Jul 09 09:55:38.683886 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.683822 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" event={"ID":"0e025dbb-4bca-492f-a806-03b1575516bb","Type":"ContainerStarted","Data":"8b45b14887a83092879dc03ce131c6e655e358dc1c464d8fec43b9d8a4525404"} Jul 09 09:55:38.685469 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.685450 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-hxqvh" event={"ID":"4e7b2884-a6b1-48ab-a959-e360e4fa153a","Type":"ContainerStarted","Data":"9aee71a827a53cd44383db53889ee70dd0eafbaa68a69dcbec18328ae05b0017"} Jul 09 09:55:38.756344 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.756297 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:38.917403 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:38.917256 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:39.018094 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.017984 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:39.338478 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.337648 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:55:39.343900 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:55:39.343873 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0666be5_86f1_4d8b_bab7_2a6eca4346f0.slice/crio-4f031517b5cec24e2cf07f02b886c9868c27e41652c46855e32838bff3912e90 WatchSource:0}: Error finding container 4f031517b5cec24e2cf07f02b886c9868c27e41652c46855e32838bff3912e90: Status 404 returned error can't find the container with id 4f031517b5cec24e2cf07f02b886c9868c27e41652c46855e32838bff3912e90 Jul 09 09:55:39.618422 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.618384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:39.680868 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.680718 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:39.694353 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.693764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" event={"ID":"a6d6fbd1-3bac-402a-91b9-45ddeaca2257","Type":"ContainerStarted","Data":"996321df2cd4e2460f5b9780ae0a48bd630494d06eebadaacc69e83c49830566"} Jul 09 09:55:39.697477 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.696722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} Jul 09 09:55:39.697477 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.696775 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"4f031517b5cec24e2cf07f02b886c9868c27e41652c46855e32838bff3912e90"} Jul 09 09:55:39.701231 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.701205 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6"} Jul 09 09:55:39.701328 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.701278 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4"} Jul 09 09:55:39.716104 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.715809 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" podStartSLOduration=1.904791621 podStartE2EDuration="3.715791424s" totalImagesPullingTime="1.810999803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:39.714008238 +0000 UTC m=+77.912100035" watchObservedRunningTime="2026-07-09 09:55:39.715791424 +0000 UTC m=+77.913883222" Jul 09 09:55:39.831931 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.831861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:39.933567 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:39.933478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:40.707658 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:40.707634 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" exitCode=0 Jul 09 09:55:40.708022 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:40.707730 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} Jul 09 09:55:41.476303 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.476269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:41.538555 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.538506 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:41.716045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.716006 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" event={"ID":"6af3cdce-e40c-4843-af80-cff1d2939ef1","Type":"ContainerStarted","Data":"ceb3fff5f937124c9240c6af4781a4704c47f76c56ef093ebd2eac7e94a8a47a"} Jul 09 09:55:41.716516 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.716108 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" event={"ID":"6af3cdce-e40c-4843-af80-cff1d2939ef1","Type":"ContainerStarted","Data":"d3227968c9bdd8ee96c128833bfaab70a31d021e3346b20659695761f4f1dde0"} Jul 09 09:55:41.716516 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.716140 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" event={"ID":"6af3cdce-e40c-4843-af80-cff1d2939ef1","Type":"ContainerStarted","Data":"f131a0a36fc2464bb14ddfff13d23c9b1c4469b566c10ebd2d5c40ac8343d8a8"} Jul 09 09:55:41.719390 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.719361 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"11603ca9e7bdceb1a4aa29589eecb2db9384dd78e65dac22ae46e1a621aaadba"} Jul 09 09:55:41.719508 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.719428 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"48df53f88106da05fbaabcd5986ce6ea57bddf68699d3d187ce9e54ceca96a06"} Jul 09 09:55:41.719508 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.719468 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" event={"ID":"dc00daab-1825-4791-9274-ad4727a6640c","Type":"ContainerStarted","Data":"08e133125c3e581957d337e23935234fe5b532d20524ea12679ac3dad637d13f"} Jul 09 09:55:41.719508 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.719504 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:41.720993 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.720968 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" event={"ID":"0e025dbb-4bca-492f-a806-03b1575516bb","Type":"ContainerStarted","Data":"101c0f74b2c7b36338ac77b13e38421ab4ae6fe9be8331447afbf37a1f41fb95"} Jul 09 09:55:41.721430 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.721406 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:41.725364 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.725340 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d"} Jul 09 09:55:41.725471 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.725407 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a"} Jul 09 09:55:41.725471 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.725448 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa"} Jul 09 09:55:41.725578 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.725494 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerStarted","Data":"60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69"} Jul 09 09:55:41.727339 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.727285 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" Jul 09 09:55:41.740545 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.740508 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7466fd7d65-fzfwp" podStartSLOduration=2.017180017 podStartE2EDuration="4.740495897s" totalImagesPullingTime="2.72331588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:41.738997798 +0000 UTC m=+79.937089597" watchObservedRunningTime="2026-07-09 09:55:41.740495897 +0000 UTC m=+79.938587693" Jul 09 09:55:41.768335 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.768290 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.41563659 podStartE2EDuration="8.768276877s" totalImagesPullingTime="4.337366839s" totalInitContainerRuntime="15.273448ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:41.765667247 +0000 UTC m=+79.963759044" watchObservedRunningTime="2026-07-09 09:55:41.768276877 +0000 UTC m=+79.966368674" Jul 09 09:55:41.789644 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.789601 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" podStartSLOduration=2.51559268 podStartE2EDuration="6.789587376s" totalImagesPullingTime="4.273994696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:41.789276186 +0000 UTC m=+79.987367986" watchObservedRunningTime="2026-07-09 09:55:41.789587376 +0000 UTC m=+79.987679174" Jul 09 09:55:41.807756 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:41.807300 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-78dh4" podStartSLOduration=2.910812839 podStartE2EDuration="5.80728708s" totalImagesPullingTime="2.896474241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:41.805262985 +0000 UTC m=+80.003354783" watchObservedRunningTime="2026-07-09 09:55:41.80728708 +0000 UTC m=+80.005378878" Jul 09 09:55:42.741727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:42.741072 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d8cddbb6f-4rm6j" Jul 09 09:55:44.158794 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:44.158697 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:44.259608 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:44.259564 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:45.753191 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.753143 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} Jul 09 09:55:45.753571 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.753281 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} Jul 09 09:55:45.753571 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.753328 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} Jul 09 09:55:45.753571 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.753370 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} Jul 09 09:55:45.753571 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.753409 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} Jul 09 09:55:45.787306 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.787272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:45.892226 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:45.892134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:46.763086 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:46.763048 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerStarted","Data":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} Jul 09 09:55:46.798747 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:46.798691 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.157102528 podStartE2EDuration="8.798669837s" totalImagesPullingTime="4.616278118s" totalInitContainerRuntime="25.289191ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:46.796411382 +0000 UTC m=+84.994503184" watchObservedRunningTime="2026-07-09 09:55:46.798669837 +0000 UTC m=+84.996761638" Jul 09 09:55:48.665399 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:48.665370 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:55:50.583033 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:50.582958 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:50.676215 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:50.676171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:51.288045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:51.288005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:51.395247 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:51.395212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:51.458865 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:51.458823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:52.205581 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:52.205537 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:52.269050 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:52.269017 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:53.118565 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:53.118529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:53.218648 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:53.218606 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:53.286301 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:53.286263 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:56.180699 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.180655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:56.281164 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.281121 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:55:56.674757 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.674731 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:56.674901 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.674782 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:55:56.810873 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.810834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-hxqvh" event={"ID":"4e7b2884-a6b1-48ab-a959-e360e4fa153a","Type":"ContainerStarted","Data":"3f67822bbb539556d637766b15f7afbaeb9f871fd3c9df88600b1998c874e92d"} Jul 09 09:55:56.811065 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.811046 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:55:56.812314 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.812293 2576 patch_prober.go:28] interesting pod/downloads-685bfb5f64-hxqvh container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.17:8080/\": dial tcp 10.134.0.17:8080: connect: connection refused" start-of-body= Jul 09 09:55:56.812385 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.812338 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-hxqvh" podUID="4e7b2884-a6b1-48ab-a959-e360e4fa153a" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.17:8080/\": dial tcp 10.134.0.17:8080: connect: connection refused" Jul 09 09:55:56.833560 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:56.833509 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-hxqvh" podStartSLOduration=2.400712815 podStartE2EDuration="20.833491653s" totalImagesPullingTime="18.432778838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:56.830974073 +0000 UTC m=+95.029065870" watchObservedRunningTime="2026-07-09 09:55:56.833491653 +0000 UTC m=+95.031583449" Jul 09 09:55:57.819980 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:55:57.819943 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-hxqvh" Jul 09 09:56:00.790315 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:00.790276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:00.890634 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:00.890592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:04.649690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:04.649657 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xdxhj" Jul 09 09:56:07.803548 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:07.803511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:07.879399 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:07.879371 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:15.781045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:15.781010 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:15.880647 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:15.880618 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:16.680723 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:16.680689 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:56:16.685265 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:16.685234 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7bc595d68f-jvnrv" Jul 09 09:56:30.853130 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:30.853092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:30.917981 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:30.917932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:32.834614 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:32.834578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:32.923897 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:32.923861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:38.665055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:38.665020 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:38.735239 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:38.735211 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:38.998515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:38.998491 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:48.187825 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:48.187788 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:48.246099 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:48.246070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:52.524446 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524414 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:56:52.524832 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524708 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="alertmanager" containerID="cri-o://68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4" gracePeriod=120 Jul 09 09:56:52.524832 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524786 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy" containerID="cri-o://73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa" gracePeriod=120 Jul 09 09:56:52.524832 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524765 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-metric" containerID="cri-o://8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a" gracePeriod=120 Jul 09 09:56:52.524832 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524820 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="prom-label-proxy" containerID="cri-o://3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d" gracePeriod=120 Jul 09 09:56:52.525020 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524834 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="config-reloader" containerID="cri-o://54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6" gracePeriod=120 Jul 09 09:56:52.525020 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:52.524777 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-web" containerID="cri-o://60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69" gracePeriod=120 Jul 09 09:56:53.043289 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043256 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d" exitCode=0 Jul 09 09:56:53.043289 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043284 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a" exitCode=0 Jul 09 09:56:53.043289 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043291 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa" exitCode=0 Jul 09 09:56:53.043289 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043296 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6" exitCode=0 Jul 09 09:56:53.043611 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043301 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4" exitCode=0 Jul 09 09:56:53.043611 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043331 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d"} Jul 09 09:56:53.043611 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043446 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a"} Jul 09 09:56:53.043611 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043490 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa"} Jul 09 09:56:53.043611 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043532 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6"} Jul 09 09:56:53.043611 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.043572 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4"} Jul 09 09:56:53.764563 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.764540 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:53.814283 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814257 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-web-config\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814283 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814287 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-volume\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814305 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-out\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814328 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-tls-assets\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814362 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814386 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-cluster-tls-config\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814412 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-main-db\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814442 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814496 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814473 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814860 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814515 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-metrics-client-ca\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814860 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814581 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clncf\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-kube-api-access-clncf\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814860 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814638 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.814860 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814676 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-web\") pod \"bd8a2cbd-ac13-4562-8f02-d9189f427578\" (UID: \"bd8a2cbd-ac13-4562-8f02-d9189f427578\") " Jul 09 09:56:53.815055 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.814873 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 09:56:53.815112 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.815089 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:53.815870 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.815808 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:53.818383 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.818296 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.818383 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.818311 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-kube-api-access-clncf" (OuterVolumeSpecName: "kube-api-access-clncf") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "kube-api-access-clncf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:56:53.818578 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.818455 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:56:53.819257 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.819216 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-volume" (OuterVolumeSpecName: "config-volume") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.819364 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.819275 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-out" (OuterVolumeSpecName: "config-out") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 09:56:53.819947 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.819919 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.820045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.819920 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.820289 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.820269 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.823829 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.823803 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.830358 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.830335 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-web-config" (OuterVolumeSpecName: "web-config") pod "bd8a2cbd-ac13-4562-8f02-d9189f427578" (UID: "bd8a2cbd-ac13-4562-8f02-d9189f427578"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915801 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clncf\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-kube-api-access-clncf\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915832 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915844 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915854 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-web-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915862 2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-volume\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915870 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-config-out\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.915869 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915877 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd8a2cbd-ac13-4562-8f02-d9189f427578-tls-assets\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.916135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915884 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-main-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.916135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915893 2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-cluster-tls-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.916135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915900 2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-main-db\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.916135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915909 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd8a2cbd-ac13-4562-8f02-d9189f427578-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.916135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915917 2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:53.916135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:53.915926 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8a2cbd-ac13-4562-8f02-d9189f427578-metrics-client-ca\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:54.050460 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.050432 2576 generic.go:356] "Generic (PLEG): container finished" podID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerID="60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69" exitCode=0 Jul 09 09:56:54.050564 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.050509 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.050564 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.050514 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69"} Jul 09 09:56:54.050670 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.050583 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd8a2cbd-ac13-4562-8f02-d9189f427578","Type":"ContainerDied","Data":"5685745a13b14d7f497ba9448e070d2432a020418e6a0ee260deb7f1672c253a"} Jul 09 09:56:54.050670 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.050617 2576 scope.go:117] "RemoveContainer" containerID="3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d" Jul 09 09:56:54.058989 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.058972 2576 scope.go:117] "RemoveContainer" containerID="8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a" Jul 09 09:56:54.065905 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.065888 2576 scope.go:117] "RemoveContainer" containerID="73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa" Jul 09 09:56:54.072227 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.072210 2576 scope.go:117] "RemoveContainer" containerID="60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69" Jul 09 09:56:54.078249 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.078228 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:56:54.079251 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.079231 2576 scope.go:117] "RemoveContainer" containerID="54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6" Jul 09 09:56:54.083463 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.083443 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:56:54.086397 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.086380 2576 scope.go:117] "RemoveContainer" containerID="68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4" Jul 09 09:56:54.092822 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.092805 2576 scope.go:117] "RemoveContainer" containerID="1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c" Jul 09 09:56:54.099356 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.099339 2576 scope.go:117] "RemoveContainer" containerID="3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d" Jul 09 09:56:54.099603 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.099585 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d\": container with ID starting with 3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d not found: ID does not exist" containerID="3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d" Jul 09 09:56:54.099691 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.099611 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d"} err="failed to get container status \"3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d\": rpc error: code = NotFound desc = could not find container \"3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d\": container with ID starting with 3ba384c04675fcb48a9ace60f1c7cd4304a8c4cd91d79fe10486223b2e03423d not found: ID does not exist" Jul 09 09:56:54.099691 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.099639 2576 scope.go:117] "RemoveContainer" containerID="8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a" Jul 09 09:56:54.099864 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.099847 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a\": container with ID starting with 8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a not found: ID does not exist" containerID="8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a" Jul 09 09:56:54.099900 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.099873 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a"} err="failed to get container status \"8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a\": rpc error: code = NotFound desc = could not find container \"8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a\": container with ID starting with 8cc312650f34d67e441eb144e65b88f6ee4be0734789ee24bc9b4c2016ac646a not found: ID does not exist" Jul 09 09:56:54.099900 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.099887 2576 scope.go:117] "RemoveContainer" containerID="73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa" Jul 09 09:56:54.100071 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.100053 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa\": container with ID starting with 73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa not found: ID does not exist" containerID="73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa" Jul 09 09:56:54.100112 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100074 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa"} err="failed to get container status \"73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa\": rpc error: code = NotFound desc = could not find container \"73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa\": container with ID starting with 73e14b9a7817027519e25d7571eec02124291930c44b5d28727e69ee7acc76aa not found: ID does not exist" Jul 09 09:56:54.100112 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100085 2576 scope.go:117] "RemoveContainer" containerID="60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69" Jul 09 09:56:54.100340 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.100323 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69\": container with ID starting with 60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69 not found: ID does not exist" containerID="60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69" Jul 09 09:56:54.100380 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100345 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69"} err="failed to get container status \"60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69\": rpc error: code = NotFound desc = could not find container \"60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69\": container with ID starting with 60e7940cbf48e82dc06ab4958b2aaee731bccd7f58f69af8db4e5077b3106e69 not found: ID does not exist" Jul 09 09:56:54.100380 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100358 2576 scope.go:117] "RemoveContainer" containerID="54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6" Jul 09 09:56:54.100605 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.100587 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6\": container with ID starting with 54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6 not found: ID does not exist" containerID="54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6" Jul 09 09:56:54.100645 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100612 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6"} err="failed to get container status \"54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6\": rpc error: code = NotFound desc = could not find container \"54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6\": container with ID starting with 54480b26a7fd92a12f4dfe8bb6c64c5418227638fd6de584730638eff7f6b2d6 not found: ID does not exist" Jul 09 09:56:54.100645 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100626 2576 scope.go:117] "RemoveContainer" containerID="68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4" Jul 09 09:56:54.100831 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.100814 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4\": container with ID starting with 68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4 not found: ID does not exist" containerID="68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4" Jul 09 09:56:54.100866 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100834 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4"} err="failed to get container status \"68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4\": rpc error: code = NotFound desc = could not find container \"68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4\": container with ID starting with 68adb6eed97c6f82976fbca25fad958c68e8a34be2110444ce91b36bac70a9f4 not found: ID does not exist" Jul 09 09:56:54.100866 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.100847 2576 scope.go:117] "RemoveContainer" containerID="1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c" Jul 09 09:56:54.101044 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:54.101029 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c\": container with ID starting with 1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c not found: ID does not exist" containerID="1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c" Jul 09 09:56:54.101081 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.101047 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c"} err="failed to get container status \"1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c\": rpc error: code = NotFound desc = could not find container \"1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c\": container with ID starting with 1fbbbd03e68ddc755ba328b0d91c874f82a433a5df00117112c941c45c399e4c not found: ID does not exist" Jul 09 09:56:54.115293 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115271 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:56:54.115568 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115557 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="init-config-reloader" Jul 09 09:56:54.115610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115570 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="init-config-reloader" Jul 09 09:56:54.115610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115582 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="alertmanager" Jul 09 09:56:54.115610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115588 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="alertmanager" Jul 09 09:56:54.115610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115596 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-metric" Jul 09 09:56:54.115610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115602 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-metric" Jul 09 09:56:54.115610 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115611 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-web" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115616 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-web" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115627 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="config-reloader" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115632 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="config-reloader" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115639 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115644 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115651 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="prom-label-proxy" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115656 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="prom-label-proxy" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115696 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="prom-label-proxy" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115705 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="config-reloader" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115710 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-web" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115716 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy-metric" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115724 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="kube-rbac-proxy" Jul 09 09:56:54.115778 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.115730 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" containerName="alertmanager" Jul 09 09:56:54.120334 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.120317 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.123213 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123194 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 09:56:54.123293 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123280 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 09:56:54.123333 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123291 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mlzxw\"" Jul 09 09:56:54.123449 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123435 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 09:56:54.123513 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123444 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 09:56:54.123513 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123439 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 09:56:54.123608 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123448 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 09:56:54.123608 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123439 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 09:56:54.123608 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.123578 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 09:56:54.128145 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.128129 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 09:56:54.134947 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.134928 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:56:54.219255 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/10051a97-9ad4-44e5-9309-eb725e42083c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219372 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219372 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/10051a97-9ad4-44e5-9309-eb725e42083c-config-out\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219372 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10051a97-9ad4-44e5-9309-eb725e42083c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219473 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219473 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-config-volume\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219473 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219421 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219473 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-web-config\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219584 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219471 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/10051a97-9ad4-44e5-9309-eb725e42083c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219584 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219584 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219584 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219560 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10051a97-9ad4-44e5-9309-eb725e42083c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.219584 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.219578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszx7\" (UniqueName: \"kubernetes.io/projected/10051a97-9ad4-44e5-9309-eb725e42083c-kube-api-access-rszx7\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320719 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10051a97-9ad4-44e5-9309-eb725e42083c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320846 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320727 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320846 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320743 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-config-volume\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320846 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320767 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320846 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320782 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-web-config\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320846 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/10051a97-9ad4-44e5-9309-eb725e42083c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.320846 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320832 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321123 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320868 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321123 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.320911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10051a97-9ad4-44e5-9309-eb725e42083c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321259 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321204 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/10051a97-9ad4-44e5-9309-eb725e42083c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321560 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321531 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10051a97-9ad4-44e5-9309-eb725e42083c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321665 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321587 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszx7\" (UniqueName: \"kubernetes.io/projected/10051a97-9ad4-44e5-9309-eb725e42083c-kube-api-access-rszx7\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321721 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321661 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/10051a97-9ad4-44e5-9309-eb725e42083c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321721 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321824 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/10051a97-9ad4-44e5-9309-eb725e42083c-config-out\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.321883 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.321819 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10051a97-9ad4-44e5-9309-eb725e42083c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.323998 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.323969 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-web-config\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.323998 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.323991 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.324144 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.324046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.324371 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.324343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/10051a97-9ad4-44e5-9309-eb725e42083c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.324475 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.324424 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-config-volume\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.324542 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.324473 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.324597 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.324567 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.324597 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.324586 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/10051a97-9ad4-44e5-9309-eb725e42083c-config-out\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.325517 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.325499 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/10051a97-9ad4-44e5-9309-eb725e42083c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.329139 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.329118 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8a2cbd-ac13-4562-8f02-d9189f427578" path="/var/lib/kubelet/pods/bd8a2cbd-ac13-4562-8f02-d9189f427578/volumes" Jul 09 09:56:54.332671 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.332653 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszx7\" (UniqueName: \"kubernetes.io/projected/10051a97-9ad4-44e5-9309-eb725e42083c-kube-api-access-rszx7\") pod \"alertmanager-main-0\" (UID: \"10051a97-9ad4-44e5-9309-eb725e42083c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.428891 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.428872 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 09:56:54.561064 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:54.561019 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 09:56:54.563238 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:56:54.563211 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10051a97_9ad4_44e5_9309_eb725e42083c.slice/crio-dd718faeac35a6c7f576f29368ca5e3c14eec8548ded1830707b9613c585e2cb WatchSource:0}: Error finding container dd718faeac35a6c7f576f29368ca5e3c14eec8548ded1830707b9613c585e2cb: Status 404 returned error can't find the container with id dd718faeac35a6c7f576f29368ca5e3c14eec8548ded1830707b9613c585e2cb Jul 09 09:56:55.056290 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:55.056260 2576 generic.go:356] "Generic (PLEG): container finished" podID="10051a97-9ad4-44e5-9309-eb725e42083c" containerID="7fd0006eff389af8dd11aa8b23cbd4e6d8106b861d583d3126bbfa222ce35b10" exitCode=0 Jul 09 09:56:55.056659 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:55.056345 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerDied","Data":"7fd0006eff389af8dd11aa8b23cbd4e6d8106b861d583d3126bbfa222ce35b10"} Jul 09 09:56:55.056659 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:55.056413 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"dd718faeac35a6c7f576f29368ca5e3c14eec8548ded1830707b9613c585e2cb"} Jul 09 09:56:56.063548 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.063516 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"158c616c0fdda51f7f79bbb4b7da63655b09c37252f1131cb8134c444f1f7a7f"} Jul 09 09:56:56.063903 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.063619 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"f0d9b20cea0891e52855c3b8c177c661975622d33ac9979921629452e7273807"} Jul 09 09:56:56.063903 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.063648 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"a80bc51149d430e3aed7cf5658e1a495352b49519517f9cbeb77fb483e1cec4e"} Jul 09 09:56:56.063903 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.063712 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"d377fa03aba2e9ce6f1d3b12ed20a248ab745296d690916048d5ae5296bd21b2"} Jul 09 09:56:56.063903 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.063736 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"09d7ee93d55aa815d5b8661aece49a98b31d55ed40943c6189fd5fe3eb1d275b"} Jul 09 09:56:56.063903 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.063760 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"10051a97-9ad4-44e5-9309-eb725e42083c","Type":"ContainerStarted","Data":"266ffa904946689fc3e9af848bd90ce48fb888096e07b82daaa0c8bfdcbb0a8f"} Jul 09 09:56:56.100100 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.100051 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.080742692 podStartE2EDuration="2.100036776s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.294084ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:56:56.099490732 +0000 UTC m=+154.297582555" watchObservedRunningTime="2026-07-09 09:56:56.100036776 +0000 UTC m=+154.298128572" Jul 09 09:56:56.770012 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.769975 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:56:56.770373 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.770343 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="prometheus" containerID="cri-o://daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" gracePeriod=600 Jul 09 09:56:56.770545 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.770520 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-web" containerID="cri-o://4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" gracePeriod=600 Jul 09 09:56:56.770613 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.770528 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy" containerID="cri-o://b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" gracePeriod=600 Jul 09 09:56:56.770672 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.770619 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-thanos" containerID="cri-o://60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" gracePeriod=600 Jul 09 09:56:56.770672 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.770641 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="thanos-sidecar" containerID="cri-o://3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" gracePeriod=600 Jul 09 09:56:56.770763 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:56.770722 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="config-reloader" containerID="cri-o://48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" gracePeriod=600 Jul 09 09:56:57.021500 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.021426 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.041941 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.041916 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-0\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.041941 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.041946 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-metrics-client-ca\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.041976 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-grpc-tls\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042002 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-serving-certs-ca-bundle\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042030 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-kube-rbac-proxy\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042054 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x7bp\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-kube-api-access-7x7bp\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042085 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042118 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-web-config\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042181 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042147 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-thanos-prometheus-http-client-file\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042244 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-db\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042272 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-metrics-client-certs\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042303 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-trusted-ca-bundle\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042331 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-tls\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042395 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config-out\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042427 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-tls-assets\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042461 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042490 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042539 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042515 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-2\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042978 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042544 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-1\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.042978 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042579 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-kubelet-serving-ca-bundle\") pod \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\" (UID: \"a0666be5-86f1-4d8b-bab7-2a6eca4346f0\") " Jul 09 09:56:57.043408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042328 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.043408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.043104 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.043408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.042365 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.043408 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.043223 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.044581 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.044337 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.044581 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.044503 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.046061 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.045242 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:56:57.046061 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.045303 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 09:56:57.046061 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.045345 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050068 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050043 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050186 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050048 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-kube-api-access-7x7bp" (OuterVolumeSpecName: "kube-api-access-7x7bp") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "kube-api-access-7x7bp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:56:57.050186 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050102 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config-out" (OuterVolumeSpecName: "config-out") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 09:56:57.050304 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050245 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config" (OuterVolumeSpecName: "config") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050416 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050392 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050469 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050413 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050519 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050473 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050628 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050592 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050716 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050632 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.050716 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.050646 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:56:57.061459 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.061423 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-web-config" (OuterVolumeSpecName: "web-config") pod "a0666be5-86f1-4d8b-bab7-2a6eca4346f0" (UID: "a0666be5-86f1-4d8b-bab7-2a6eca4346f0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072870 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" exitCode=0 Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072900 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" exitCode=0 Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072909 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" exitCode=0 Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072918 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" exitCode=0 Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072926 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" exitCode=0 Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072934 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" exitCode=0 Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.072962 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073038 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073213 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073254 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073292 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073332 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a0666be5-86f1-4d8b-bab7-2a6eca4346f0","Type":"ContainerDied","Data":"4f031517b5cec24e2cf07f02b886c9868c27e41652c46855e32838bff3912e90"} Jul 09 09:56:57.074214 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.073381 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.086788 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.086755 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.094300 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.094282 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.101013 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.100984 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.108549 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.108520 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:56:57.109069 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.109049 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.115819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.115797 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:56:57.116664 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.116650 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.123815 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.123798 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.130200 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130175 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.130458 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.130438 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.130523 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130465 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} err="failed to get container status \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" Jul 09 09:56:57.130523 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130482 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.130722 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.130708 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.130764 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130725 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} err="failed to get container status \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" Jul 09 09:56:57.130764 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130738 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.130945 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.130930 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.130989 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130946 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} err="failed to get container status \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" Jul 09 09:56:57.130989 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.130958 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.131171 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.131140 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.131273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131174 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} err="failed to get container status \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" Jul 09 09:56:57.131273 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131187 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.131430 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.131414 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.131466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131433 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} err="failed to get container status \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" Jul 09 09:56:57.131466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131445 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.131671 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.131653 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.131735 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131680 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} err="failed to get container status \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" Jul 09 09:56:57.131735 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131702 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.131987 ip-10-0-138-234 kubenswrapper[2576]: E0709 09:56:57.131968 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.132034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.131992 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} err="failed to get container status \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" Jul 09 09:56:57.132034 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132006 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.132317 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132296 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} err="failed to get container status \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" Jul 09 09:56:57.132375 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132329 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.132564 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132547 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} err="failed to get container status \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" Jul 09 09:56:57.132606 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132566 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.132797 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132780 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} err="failed to get container status \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" Jul 09 09:56:57.132849 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132798 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.133000 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.132983 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} err="failed to get container status \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" Jul 09 09:56:57.133044 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.133000 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.133241 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.133223 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} err="failed to get container status \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" Jul 09 09:56:57.133292 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.133242 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.133446 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.133425 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} err="failed to get container status \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" Jul 09 09:56:57.133494 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.133446 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.138607 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.138582 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} err="failed to get container status \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" Jul 09 09:56:57.138694 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.138623 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.138915 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.138896 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} err="failed to get container status \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" Jul 09 09:56:57.138976 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.138916 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.139170 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139128 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} err="failed to get container status \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" Jul 09 09:56:57.139230 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139171 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.139412 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139391 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} err="failed to get container status \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" Jul 09 09:56:57.139459 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139414 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.139684 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139666 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} err="failed to get container status \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" Jul 09 09:56:57.139744 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139685 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.139984 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139950 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} err="failed to get container status \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" Jul 09 09:56:57.140037 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.139990 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.140339 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.140291 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} err="failed to get container status \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" Jul 09 09:56:57.140407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.140340 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.140707 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.140669 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} err="failed to get container status \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" Jul 09 09:56:57.140790 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.140716 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.141033 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141012 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} err="failed to get container status \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" Jul 09 09:56:57.141124 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141033 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.141252 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141235 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} err="failed to get container status \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" Jul 09 09:56:57.141297 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141253 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.141492 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141473 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} err="failed to get container status \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" Jul 09 09:56:57.141559 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141494 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.141768 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141744 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} err="failed to get container status \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" Jul 09 09:56:57.141870 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.141769 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.142039 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142019 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} err="failed to get container status \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" Jul 09 09:56:57.142138 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142040 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.142335 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142315 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} err="failed to get container status \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" Jul 09 09:56:57.142426 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142336 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.142621 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142598 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} err="failed to get container status \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" Jul 09 09:56:57.142721 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142622 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.142918 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142899 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} err="failed to get container status \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" Jul 09 09:56:57.142918 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.142918 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.143145 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143123 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} err="failed to get container status \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" Jul 09 09:56:57.143247 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143173 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.143690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143403 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} err="failed to get container status \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" Jul 09 09:56:57.143690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143425 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143692 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config-out\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143709 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-tls-assets\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143722 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143737 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143762 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143779 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143794 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143809 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143823 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-metrics-client-ca\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.143831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143835 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-grpc-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143843 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143840 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} err="failed to get container status \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143874 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143852 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-kube-rbac-proxy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143949 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x7bp\" (UniqueName: \"kubernetes.io/projected/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-kube-api-access-7x7bp\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143967 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143982 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-web-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.143997 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144011 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-k8s-db\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144026 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-metrics-client-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144039 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144053 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a0666be5-86f1-4d8b-bab7-2a6eca4346f0-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144095 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} err="failed to get container status \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144115 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144342 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} err="failed to get container status \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" Jul 09 09:56:57.144355 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144358 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.144899 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144581 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} err="failed to get container status \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" Jul 09 09:56:57.144899 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144596 2576 scope.go:117] "RemoveContainer" containerID="60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682" Jul 09 09:56:57.144899 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144806 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682"} err="failed to get container status \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": rpc error: code = NotFound desc = could not find container \"60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682\": container with ID starting with 60c29d68783b99b25ee8bd64f0b68b9dc3459e8faf1cff51940b747e35bfe682 not found: ID does not exist" Jul 09 09:56:57.144899 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.144818 2576 scope.go:117] "RemoveContainer" containerID="b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070" Jul 09 09:56:57.145045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145002 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070"} err="failed to get container status \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": rpc error: code = NotFound desc = could not find container \"b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070\": container with ID starting with b55848204f680d90e72af7feb96dbe5ff2e84b9c995a8b2d893770c461aaa070 not found: ID does not exist" Jul 09 09:56:57.145045 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145016 2576 scope.go:117] "RemoveContainer" containerID="4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea" Jul 09 09:56:57.145217 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145202 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea"} err="failed to get container status \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": rpc error: code = NotFound desc = could not find container \"4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea\": container with ID starting with 4aa161bbc11b86db7aa2a4ae8c30e15a7c13f75f67d1a94fa8f8f771212d78ea not found: ID does not exist" Jul 09 09:56:57.145217 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145217 2576 scope.go:117] "RemoveContainer" containerID="3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d" Jul 09 09:56:57.145428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145414 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d"} err="failed to get container status \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": rpc error: code = NotFound desc = could not find container \"3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d\": container with ID starting with 3d68e119e53e0487e21efe162dab0243613acce296190748854925e4b3a4ac3d not found: ID does not exist" Jul 09 09:56:57.145428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145428 2576 scope.go:117] "RemoveContainer" containerID="48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82" Jul 09 09:56:57.145658 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145640 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82"} err="failed to get container status \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": rpc error: code = NotFound desc = could not find container \"48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82\": container with ID starting with 48fec94f80c3bc30a1d631008dde0542272ee0c862605afded55cd4d2817dc82 not found: ID does not exist" Jul 09 09:56:57.145709 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145658 2576 scope.go:117] "RemoveContainer" containerID="daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6" Jul 09 09:56:57.145859 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145843 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6"} err="failed to get container status \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": rpc error: code = NotFound desc = could not find container \"daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6\": container with ID starting with daa390578dd9dcf698ac3bfe72d635ff34362c9eb071165d9d6647d4d4aeffa6 not found: ID does not exist" Jul 09 09:56:57.145902 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.145860 2576 scope.go:117] "RemoveContainer" containerID="9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a" Jul 09 09:56:57.146061 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.146045 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a"} err="failed to get container status \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": rpc error: code = NotFound desc = could not find container \"9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a\": container with ID starting with 9066dce73154503b0bf9dd714d504defb758171aedbde87f319331a5cbc3687a not found: ID does not exist" Jul 09 09:56:57.155723 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.155703 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:56:57.156145 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156122 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="config-reloader" Jul 09 09:56:57.156145 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156143 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="config-reloader" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156175 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-thanos" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156185 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-thanos" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156195 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156205 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156216 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="init-config-reloader" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156224 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="init-config-reloader" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156237 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="prometheus" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156246 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="prometheus" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156257 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-web" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156265 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-web" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156285 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="thanos-sidecar" Jul 09 09:56:57.156365 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156294 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="thanos-sidecar" Jul 09 09:56:57.158713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156380 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="prometheus" Jul 09 09:56:57.158713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156395 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="config-reloader" Jul 09 09:56:57.158713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156401 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-web" Jul 09 09:56:57.158713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156408 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy" Jul 09 09:56:57.158713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156414 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="kube-rbac-proxy-thanos" Jul 09 09:56:57.158713 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.156421 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" containerName="thanos-sidecar" Jul 09 09:56:57.162499 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.162482 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.165113 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.165093 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 09:56:57.166322 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.166288 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 09:56:57.166428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.166344 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 09:56:57.166428 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.166348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-udh7e9hts1am\"" Jul 09 09:56:57.166627 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.166609 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 09:56:57.166983 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.166967 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 09:56:57.167200 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.166987 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 09:56:57.167200 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.167062 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 09:56:57.167328 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.167202 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v8jtl\"" Jul 09 09:56:57.167328 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.167231 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 09:56:57.167831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.167776 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 09:56:57.168035 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.168021 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 09:56:57.168697 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.168682 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 09:56:57.168864 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.168848 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 09:56:57.172567 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.172547 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 09:56:57.178271 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.178253 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 09:56:57.183830 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.183808 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:56:57.213252 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.212815 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:57.244375 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcz5n\" (UniqueName: \"kubernetes.io/projected/0051aa0e-be16-489e-9939-a10fd1e26f96-kube-api-access-kcz5n\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244466 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244594 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244594 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244594 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244543 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244634 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-config\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244721 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0051aa0e-be16-489e-9939-a10fd1e26f96-config-out\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244773 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-web-config\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244992 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244992 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.244992 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.244888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0051aa0e-be16-489e-9939-a10fd1e26f96-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.296639 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.296556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:56:57.346043 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0051aa0e-be16-489e-9939-a10fd1e26f96-config-out\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346072 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346135 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346092 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346321 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346177 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-web-config\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346321 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346321 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346321 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0051aa0e-be16-489e-9939-a10fd1e26f96-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346345 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz5n\" (UniqueName: \"kubernetes.io/projected/0051aa0e-be16-489e-9939-a10fd1e26f96-kube-api-access-kcz5n\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346372 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346398 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346515 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346493 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346565 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-config\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.346856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346672 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.347167 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.346890 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.347352 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.347323 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349093 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349068 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0051aa0e-be16-489e-9939-a10fd1e26f96-config-out\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349210 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349098 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-web-config\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349567 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349266 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349567 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349278 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349567 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349488 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0051aa0e-be16-489e-9939-a10fd1e26f96-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349567 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349521 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.349822 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.349770 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.350192 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.350105 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.350296 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.350190 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.350296 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.350250 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.350407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.350335 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.350407 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.350348 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.351693 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.351665 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-config\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.352080 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.352062 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.352206 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.352189 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0051aa0e-be16-489e-9939-a10fd1e26f96-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.352244 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.352233 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0051aa0e-be16-489e-9939-a10fd1e26f96-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.359223 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.359207 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcz5n\" (UniqueName: \"kubernetes.io/projected/0051aa0e-be16-489e-9939-a10fd1e26f96-kube-api-access-kcz5n\") pod \"prometheus-k8s-0\" (UID: \"0051aa0e-be16-489e-9939-a10fd1e26f96\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.472329 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.472304 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:56:57.608718 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:57.608691 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 09:56:57.610241 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:56:57.610217 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0051aa0e_be16_489e_9939_a10fd1e26f96.slice/crio-ce65f0bcf397cc88576bcb1264958d2f6511db081d23be01baf00cf715581a3a WatchSource:0}: Error finding container ce65f0bcf397cc88576bcb1264958d2f6511db081d23be01baf00cf715581a3a: Status 404 returned error can't find the container with id ce65f0bcf397cc88576bcb1264958d2f6511db081d23be01baf00cf715581a3a Jul 09 09:56:58.080019 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:58.079985 2576 generic.go:356] "Generic (PLEG): container finished" podID="0051aa0e-be16-489e-9939-a10fd1e26f96" containerID="eacde3db445933d48eda79d68a4a859b97e9a267dada9cc907dc7c5d51d940f8" exitCode=0 Jul 09 09:56:58.080438 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:58.080063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerDied","Data":"eacde3db445933d48eda79d68a4a859b97e9a267dada9cc907dc7c5d51d940f8"} Jul 09 09:56:58.080438 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:58.080121 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"ce65f0bcf397cc88576bcb1264958d2f6511db081d23be01baf00cf715581a3a"} Jul 09 09:56:58.330210 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:58.330127 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0666be5-86f1-4d8b-bab7-2a6eca4346f0" path="/var/lib/kubelet/pods/a0666be5-86f1-4d8b-bab7-2a6eca4346f0/volumes" Jul 09 09:56:59.088218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.088183 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"b8529794b5e8817afd83a0d1423e2363322fb62ced1a63fc1e35182b73898d73"} Jul 09 09:56:59.088680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.088286 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"62b76aa1b3939f40d5041c09beb74f1600200c6552b03fa522b6a37a61df3286"} Jul 09 09:56:59.088680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.088333 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"34fa78439ea42f6cb93f8b6ec501bba043b7c312960320097e58c042b7530786"} Jul 09 09:56:59.088680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.088375 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"39bceb20e5e671d61f1b768cbd3d628380b077421f79e0cffcb4284a20f01c6b"} Jul 09 09:56:59.088680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.088413 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"f9a468a5bb6194866ac0e79a1a2927e99e46d5fa95d40778012c40ba1afc17b9"} Jul 09 09:56:59.088680 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.088450 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0051aa0e-be16-489e-9939-a10fd1e26f96","Type":"ContainerStarted","Data":"e53d5326888afa87f9bd5cb0ac6a6c52a9384147e4eea8cbcf2a1b5971365cdd"} Jul 09 09:56:59.128477 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:56:59.128408 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.107734537 podStartE2EDuration="2.128388972s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.654435ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:56:59.126690466 +0000 UTC m=+157.324782263" watchObservedRunningTime="2026-07-09 09:56:59.128388972 +0000 UTC m=+157.326480770" Jul 09 09:57:00.146965 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:00.146924 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:00.247899 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:00.247863 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:02.472754 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:02.472720 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:57:08.097636 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:08.097595 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:08.355249 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:08.355143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:15.780097 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:15.780012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:15.880141 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:15.880104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:15.948405 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:15.948377 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:36.689493 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:36.689454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:36.790933 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:36.790886 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:39.081386 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:39.081344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:57.472808 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:57.472775 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:57:57.488394 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:57.488371 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:57:58.319467 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:58.319425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:57:58.344672 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:58.344647 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 09:57:59.080101 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:57:59.080066 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:02.115648 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:02.115610 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:02.215387 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:02.215365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:04.196468 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:04.196431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:04.288429 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:04.288393 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:04.736426 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:04.736392 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:04.841049 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:04.841021 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:06.407636 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.407600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:06.724244 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.724210 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9c54q"] Jul 09 09:58:06.727312 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.727291 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:06.730375 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.730354 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 09:58:06.731758 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.731743 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mgpvt\"" Jul 09 09:58:06.731828 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.731760 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 09:58:06.738695 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.738673 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9c54q"] Jul 09 09:58:06.804429 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.804401 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm4zx\" (UniqueName: \"kubernetes.io/projected/96a23157-6cd2-4cbe-b37b-dc035e75179c-kube-api-access-lm4zx\") pod \"cert-manager-cainjector-559cdd8c87-9c54q\" (UID: \"96a23157-6cd2-4cbe-b37b-dc035e75179c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:06.804538 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.804471 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96a23157-6cd2-4cbe-b37b-dc035e75179c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9c54q\" (UID: \"96a23157-6cd2-4cbe-b37b-dc035e75179c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:06.874159 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.874130 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:06.905224 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.905199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96a23157-6cd2-4cbe-b37b-dc035e75179c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9c54q\" (UID: \"96a23157-6cd2-4cbe-b37b-dc035e75179c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:06.905333 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.905253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm4zx\" (UniqueName: \"kubernetes.io/projected/96a23157-6cd2-4cbe-b37b-dc035e75179c-kube-api-access-lm4zx\") pod \"cert-manager-cainjector-559cdd8c87-9c54q\" (UID: \"96a23157-6cd2-4cbe-b37b-dc035e75179c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:06.915690 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.915664 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96a23157-6cd2-4cbe-b37b-dc035e75179c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9c54q\" (UID: \"96a23157-6cd2-4cbe-b37b-dc035e75179c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:06.915910 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:06.915895 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm4zx\" (UniqueName: \"kubernetes.io/projected/96a23157-6cd2-4cbe-b37b-dc035e75179c-kube-api-access-lm4zx\") pod \"cert-manager-cainjector-559cdd8c87-9c54q\" (UID: \"96a23157-6cd2-4cbe-b37b-dc035e75179c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:07.044352 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:07.044269 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" Jul 09 09:58:07.170674 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:07.170633 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9c54q"] Jul 09 09:58:07.174280 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:58:07.174253 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96a23157_6cd2_4cbe_b37b_dc035e75179c.slice/crio-e438ede40e9c6aad67e318f738d6b1e796b75eb791e18a3b88580d28056b0f2a WatchSource:0}: Error finding container e438ede40e9c6aad67e318f738d6b1e796b75eb791e18a3b88580d28056b0f2a: Status 404 returned error can't find the container with id e438ede40e9c6aad67e318f738d6b1e796b75eb791e18a3b88580d28056b0f2a Jul 09 09:58:07.362749 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:07.362667 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" event={"ID":"96a23157-6cd2-4cbe-b37b-dc035e75179c","Type":"ContainerStarted","Data":"e438ede40e9c6aad67e318f738d6b1e796b75eb791e18a3b88580d28056b0f2a"} Jul 09 09:58:08.936552 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:08.936513 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:09.035317 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:09.035274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:11.357576 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:11.357540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:11.383857 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:11.383826 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" event={"ID":"96a23157-6cd2-4cbe-b37b-dc035e75179c","Type":"ContainerStarted","Data":"56cb889174346c47c7f1a5900038924d544d829fdb7b1e3b49c6e66a864995b9"} Jul 09 09:58:11.462102 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:11.462070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:11.819751 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:11.819714 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:14.975616 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:14.975574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:20.319242 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:20.319210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:20.415211 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:20.415182 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:22.547060 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.547014 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9c54q" podStartSLOduration=13.270837819 podStartE2EDuration="16.54699865s" totalImagesPullingTime="3.276160831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:58:11.409491224 +0000 UTC m=+229.607583021" watchObservedRunningTime="2026-07-09 09:58:22.54699865 +0000 UTC m=+240.745090446" Jul 09 09:58:22.547593 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.547578 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr"] Jul 09 09:58:22.550844 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.550829 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.554065 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.554042 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-th5ld\"" Jul 09 09:58:22.554177 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.554078 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 09:58:22.555308 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.555293 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 09:58:22.561540 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.561518 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr"] Jul 09 09:58:22.634215 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.634189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a66bc06a-438c-462a-bfda-00df45bb37ff-tmp\") pod \"openshift-lws-operator-fd8ccff4c-px2wr\" (UID: \"a66bc06a-438c-462a-bfda-00df45bb37ff\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.634327 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.634237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtswt\" (UniqueName: \"kubernetes.io/projected/a66bc06a-438c-462a-bfda-00df45bb37ff-kube-api-access-mtswt\") pod \"openshift-lws-operator-fd8ccff4c-px2wr\" (UID: \"a66bc06a-438c-462a-bfda-00df45bb37ff\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.734517 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.734488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a66bc06a-438c-462a-bfda-00df45bb37ff-tmp\") pod \"openshift-lws-operator-fd8ccff4c-px2wr\" (UID: \"a66bc06a-438c-462a-bfda-00df45bb37ff\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.734669 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.734536 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtswt\" (UniqueName: \"kubernetes.io/projected/a66bc06a-438c-462a-bfda-00df45bb37ff-kube-api-access-mtswt\") pod \"openshift-lws-operator-fd8ccff4c-px2wr\" (UID: \"a66bc06a-438c-462a-bfda-00df45bb37ff\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.734969 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.734941 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a66bc06a-438c-462a-bfda-00df45bb37ff-tmp\") pod \"openshift-lws-operator-fd8ccff4c-px2wr\" (UID: \"a66bc06a-438c-462a-bfda-00df45bb37ff\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.746781 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.746472 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtswt\" (UniqueName: \"kubernetes.io/projected/a66bc06a-438c-462a-bfda-00df45bb37ff-kube-api-access-mtswt\") pod \"openshift-lws-operator-fd8ccff4c-px2wr\" (UID: \"a66bc06a-438c-462a-bfda-00df45bb37ff\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.859119 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.859044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" Jul 09 09:58:22.987623 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:22.987599 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr"] Jul 09 09:58:22.989606 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:58:22.989578 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda66bc06a_438c_462a_bfda_00df45bb37ff.slice/crio-9d5a0c7a4e1b5bd94e9b0f99a39574f794b2e910da7c00276eff5c1573376e7f WatchSource:0}: Error finding container 9d5a0c7a4e1b5bd94e9b0f99a39574f794b2e910da7c00276eff5c1573376e7f: Status 404 returned error can't find the container with id 9d5a0c7a4e1b5bd94e9b0f99a39574f794b2e910da7c00276eff5c1573376e7f Jul 09 09:58:23.431588 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:23.431552 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" event={"ID":"a66bc06a-438c-462a-bfda-00df45bb37ff","Type":"ContainerStarted","Data":"9d5a0c7a4e1b5bd94e9b0f99a39574f794b2e910da7c00276eff5c1573376e7f"} Jul 09 09:58:27.450401 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:27.450370 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" event={"ID":"a66bc06a-438c-462a-bfda-00df45bb37ff","Type":"ContainerStarted","Data":"eb022f041af927f02df1be6229ad0768fa5bc3b653e515aae6a21056b0bb921b"} Jul 09 09:58:27.473437 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:27.473391 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-px2wr" podStartSLOduration=2.030821572 podStartE2EDuration="5.473376557s" totalImagesPullingTime="3.442554985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:58:27.472442451 +0000 UTC m=+245.670534249" watchObservedRunningTime="2026-07-09 09:58:27.473376557 +0000 UTC m=+245.671468353" Jul 09 09:58:28.146921 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:28.146879 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:28.253103 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:28.253073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:30.379684 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:30.379646 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:46.115815 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:46.115735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:46.234751 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:46.234720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:46.730762 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:46.730720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:47.145627 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.145553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:47.263317 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.263271 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:47.474812 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.474778 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk"] Jul 09 09:58:47.478041 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.478026 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.486429 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.486406 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 09:58:47.486561 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.486463 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zsbc7\"" Jul 09 09:58:47.487192 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.487176 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 09:58:47.487339 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.487321 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 09:58:47.487505 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.487487 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 09:58:47.591309 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.591279 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk"] Jul 09 09:58:47.637729 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.637688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c81b79de-11aa-4b6e-bfc9-bf85663de440-tmp\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.637908 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.637744 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c81b79de-11aa-4b6e-bfc9-bf85663de440-manifests\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.637908 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.637771 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c81b79de-11aa-4b6e-bfc9-bf85663de440-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.637908 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.637810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c81b79de-11aa-4b6e-bfc9-bf85663de440-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.637908 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.637866 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt2th\" (UniqueName: \"kubernetes.io/projected/c81b79de-11aa-4b6e-bfc9-bf85663de440-kube-api-access-gt2th\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.738756 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.738651 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c81b79de-11aa-4b6e-bfc9-bf85663de440-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.738756 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.738720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c81b79de-11aa-4b6e-bfc9-bf85663de440-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.738985 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.738762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt2th\" (UniqueName: \"kubernetes.io/projected/c81b79de-11aa-4b6e-bfc9-bf85663de440-kube-api-access-gt2th\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.738985 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.738837 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c81b79de-11aa-4b6e-bfc9-bf85663de440-tmp\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.739362 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.739138 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c81b79de-11aa-4b6e-bfc9-bf85663de440-manifests\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.739586 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.739558 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c81b79de-11aa-4b6e-bfc9-bf85663de440-manifests\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.741368 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.741340 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c81b79de-11aa-4b6e-bfc9-bf85663de440-tmp\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.741471 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.741461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c81b79de-11aa-4b6e-bfc9-bf85663de440-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.741593 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.741573 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c81b79de-11aa-4b6e-bfc9-bf85663de440-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.754647 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.754616 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt2th\" (UniqueName: \"kubernetes.io/projected/c81b79de-11aa-4b6e-bfc9-bf85663de440-kube-api-access-gt2th\") pod \"opendatahub-operator-controller-manager-5cb598dddb-f8nlk\" (UID: \"c81b79de-11aa-4b6e-bfc9-bf85663de440\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.790538 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.790513 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:47.825722 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.825693 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:47.918528 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.918490 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:47.938848 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:47.938820 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk"] Jul 09 09:58:47.941804 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:58:47.941780 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81b79de_11aa_4b6e_bfc9_bf85663de440.slice/crio-395cc31223c40139008d654927b608b16bb7fe8e2f40710fca74de69708caf1d WatchSource:0}: Error finding container 395cc31223c40139008d654927b608b16bb7fe8e2f40710fca74de69708caf1d: Status 404 returned error can't find the container with id 395cc31223c40139008d654927b608b16bb7fe8e2f40710fca74de69708caf1d Jul 09 09:58:48.021589 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.021527 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:48.127816 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.127793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:48.360367 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.360307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:48.462193 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.462144 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:48.536119 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.536092 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" event={"ID":"c81b79de-11aa-4b6e-bfc9-bf85663de440","Type":"ContainerStarted","Data":"395cc31223c40139008d654927b608b16bb7fe8e2f40710fca74de69708caf1d"} Jul 09 09:58:48.566257 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.566228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:48.675946 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:48.675872 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:51.551716 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:51.551679 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" event={"ID":"c81b79de-11aa-4b6e-bfc9-bf85663de440","Type":"ContainerStarted","Data":"1151e96078414cdf902a5ec5378f5eb27b890bab7d3e5f9dff7636e1df7ffb70"} Jul 09 09:58:52.557561 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:52.557525 2576 generic.go:356] "Generic (PLEG): container finished" podID="c81b79de-11aa-4b6e-bfc9-bf85663de440" containerID="1151e96078414cdf902a5ec5378f5eb27b890bab7d3e5f9dff7636e1df7ffb70" exitCode=0 Jul 09 09:58:52.557907 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:52.557610 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" event={"ID":"c81b79de-11aa-4b6e-bfc9-bf85663de440","Type":"ContainerDied","Data":"1151e96078414cdf902a5ec5378f5eb27b890bab7d3e5f9dff7636e1df7ffb70"} Jul 09 09:58:53.564218 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:53.564190 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" event={"ID":"c81b79de-11aa-4b6e-bfc9-bf85663de440","Type":"ContainerStarted","Data":"9915e0f1d63802d0a30a67eecaafdbb866835a801e59cc1ed5d697004d172acf"} Jul 09 09:58:53.564586 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:53.564258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:58:53.605770 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:53.605722 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" podStartSLOduration=2.035102329 podStartE2EDuration="6.605706223s" totalImagesPullingTime="3.490298903s" totalInitContainerRuntime="1.080304991s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:58:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:58:53.605247242 +0000 UTC m=+271.803339039" watchObservedRunningTime="2026-07-09 09:58:53.605706223 +0000 UTC m=+271.803798021" Jul 09 09:58:58.715385 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:58.715353 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:58:58.812144 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:58:58.812112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:01.297682 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:01.297637 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:01.569980 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:01.569897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:04.570743 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:04.570717 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-f8nlk" Jul 09 09:59:08.399252 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:08.399221 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:11.085636 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:11.085598 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:12.187429 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:12.187395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:12.283663 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:12.283631 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:14.737286 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:14.737252 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:15.740195 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.740160 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm"] Jul 09 09:59:15.743376 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.743361 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.758864 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.758843 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lkvqp\"" Jul 09 09:59:15.759278 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.759263 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 09:59:15.759369 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.759348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 09:59:15.763527 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.763508 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm"] Jul 09 09:59:15.765800 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.765782 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 09:59:15.793566 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.793543 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbhqf\" (UniqueName: \"kubernetes.io/projected/2514eb6f-afde-4520-a3fd-e3f16dee83d0-kube-api-access-nbhqf\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.793651 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.793574 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2514eb6f-afde-4520-a3fd-e3f16dee83d0-metrics-cert\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.793651 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.793605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2514eb6f-afde-4520-a3fd-e3f16dee83d0-cert\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.793725 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.793697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2514eb6f-afde-4520-a3fd-e3f16dee83d0-manager-config\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.894834 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.894807 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbhqf\" (UniqueName: \"kubernetes.io/projected/2514eb6f-afde-4520-a3fd-e3f16dee83d0-kube-api-access-nbhqf\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.894948 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.894838 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2514eb6f-afde-4520-a3fd-e3f16dee83d0-metrics-cert\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.894948 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.894866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2514eb6f-afde-4520-a3fd-e3f16dee83d0-cert\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.894948 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.894905 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2514eb6f-afde-4520-a3fd-e3f16dee83d0-manager-config\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.895647 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.895621 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2514eb6f-afde-4520-a3fd-e3f16dee83d0-manager-config\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.897490 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.897464 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2514eb6f-afde-4520-a3fd-e3f16dee83d0-metrics-cert\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.897597 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.897497 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2514eb6f-afde-4520-a3fd-e3f16dee83d0-cert\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:15.955639 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:15.955613 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbhqf\" (UniqueName: \"kubernetes.io/projected/2514eb6f-afde-4520-a3fd-e3f16dee83d0-kube-api-access-nbhqf\") pod \"lws-controller-manager-5c844974c9-r7rvm\" (UID: \"2514eb6f-afde-4520-a3fd-e3f16dee83d0\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:16.052285 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:16.052224 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:16.324024 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:16.323996 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm"] Jul 09 09:59:16.325133 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:59:16.325112 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2514eb6f_afde_4520_a3fd_e3f16dee83d0.slice/crio-03d3f59a0bf36f20588f2ff9a26d381f3354b4251e4a87cf6c1e9e753ad81087 WatchSource:0}: Error finding container 03d3f59a0bf36f20588f2ff9a26d381f3354b4251e4a87cf6c1e9e753ad81087: Status 404 returned error can't find the container with id 03d3f59a0bf36f20588f2ff9a26d381f3354b4251e4a87cf6c1e9e753ad81087 Jul 09 09:59:16.665964 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:16.665883 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" event={"ID":"2514eb6f-afde-4520-a3fd-e3f16dee83d0","Type":"ContainerStarted","Data":"03d3f59a0bf36f20588f2ff9a26d381f3354b4251e4a87cf6c1e9e753ad81087"} Jul 09 09:59:18.492038 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:18.492002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:18.626147 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:18.626116 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:18.675981 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:18.675948 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" event={"ID":"2514eb6f-afde-4520-a3fd-e3f16dee83d0","Type":"ContainerStarted","Data":"2a26b18490ee5586a78f4e58df568cfa3f7cff2fc80944b0e1b0d6ca7d4733f6"} Jul 09 09:59:18.676183 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:18.676063 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:18.764532 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:18.764433 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" podStartSLOduration=2.037219008 podStartE2EDuration="3.76441752s" totalImagesPullingTime="1.727198512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:18.752088407 +0000 UTC m=+296.950180204" watchObservedRunningTime="2026-07-09 09:59:18.76441752 +0000 UTC m=+296.962509317" Jul 09 09:59:19.086860 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.086784 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r"] Jul 09 09:59:19.090117 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.090101 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.093224 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.093199 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tf55z\"" Jul 09 09:59:19.094311 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.094288 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 09:59:19.094413 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.094290 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 09:59:19.094413 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.094290 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 09:59:19.103317 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.103295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:19.115290 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.115270 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r"] Jul 09 09:59:19.224696 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224673 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224756 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224821 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224814 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224965 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224965 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224965 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224919 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.224965 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.224937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf5zk\" (UniqueName: \"kubernetes.io/projected/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-kube-api-access-nf5zk\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326193 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326322 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326218 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326334 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326393 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326507 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf5zk\" (UniqueName: \"kubernetes.io/projected/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-kube-api-access-nf5zk\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326507 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326438 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326507 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326662 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326547 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326662 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326777 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326722 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326774 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.326937 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.326916 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.327073 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.327013 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.327247 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.327229 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.328556 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.328537 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.328819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.328799 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.345573 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.345524 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf5zk\" (UniqueName: \"kubernetes.io/projected/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-kube-api-access-nf5zk\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.355476 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.355452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:19.433443 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.433419 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c262fcfc-ee30-4f3a-8de5-e23a6c96f86b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678vnp2r\" (UID: \"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.564831 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.564797 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:19.700253 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.700181 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:19.863816 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:19.863795 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r"] Jul 09 09:59:19.865939 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:59:19.865911 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc262fcfc_ee30_4f3a_8de5_e23a6c96f86b.slice/crio-68fee976fa94a583d31c702dd41ad42ca7a1add75c2560e0afa57d30b16f5e83 WatchSource:0}: Error finding container 68fee976fa94a583d31c702dd41ad42ca7a1add75c2560e0afa57d30b16f5e83: Status 404 returned error can't find the container with id 68fee976fa94a583d31c702dd41ad42ca7a1add75c2560e0afa57d30b16f5e83 Jul 09 09:59:20.689259 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:20.689220 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" event={"ID":"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b","Type":"ContainerStarted","Data":"68fee976fa94a583d31c702dd41ad42ca7a1add75c2560e0afa57d30b16f5e83"} Jul 09 09:59:21.322988 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.322950 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl"] Jul 09 09:59:21.330898 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.330876 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.333878 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.333796 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 09:59:21.333878 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.333839 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 09:59:21.333878 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.333872 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rlxbb\"" Jul 09 09:59:21.339682 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.339511 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 09:59:21.342825 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.342766 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl"] Jul 09 09:59:21.446543 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446515 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.446730 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4257f47c-2113-4728-97c3-2089b6266d59-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.446730 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4257f47c-2113-4728-97c3-2089b6266d59-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.446730 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446637 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.446730 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4257f47c-2113-4728-97c3-2089b6266d59-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.446912 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446779 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.446912 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.446833 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfrlt\" (UniqueName: \"kubernetes.io/projected/4257f47c-2113-4728-97c3-2089b6266d59-kube-api-access-sfrlt\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.547865 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.547830 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfrlt\" (UniqueName: \"kubernetes.io/projected/4257f47c-2113-4728-97c3-2089b6266d59-kube-api-access-sfrlt\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548038 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.547915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548038 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.547954 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4257f47c-2113-4728-97c3-2089b6266d59-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548038 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.547992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4257f47c-2113-4728-97c3-2089b6266d59-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548038 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.548025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548335 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.548064 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4257f47c-2113-4728-97c3-2089b6266d59-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548335 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.548111 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.548927 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.548905 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4257f47c-2113-4728-97c3-2089b6266d59-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.550814 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.550786 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.550923 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.550816 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4257f47c-2113-4728-97c3-2089b6266d59-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.551056 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.551036 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.551131 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.551056 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4257f47c-2113-4728-97c3-2089b6266d59-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.564865 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.564844 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4257f47c-2113-4728-97c3-2089b6266d59-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.569535 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.569516 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfrlt\" (UniqueName: \"kubernetes.io/projected/4257f47c-2113-4728-97c3-2089b6266d59-kube-api-access-sfrlt\") pod \"istiod-openshift-gateway-94bb8fbfd-kmbnl\" (UID: \"4257f47c-2113-4728-97c3-2089b6266d59\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:21.643502 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:21.643433 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:22.373631 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:22.373506 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 09:59:22.620856 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:22.496548 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl"] Jul 09 09:59:22.621508 ip-10-0-138-234 kubenswrapper[2576]: W0709 09:59:22.621477 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4257f47c_2113_4728_97c3_2089b6266d59.slice/crio-028e54cdc408496fc1140e9c0b835228a9c032ea4e6bbbeecdb25ad61d8222ca WatchSource:0}: Error finding container 028e54cdc408496fc1140e9c0b835228a9c032ea4e6bbbeecdb25ad61d8222ca: Status 404 returned error can't find the container with id 028e54cdc408496fc1140e9c0b835228a9c032ea4e6bbbeecdb25ad61d8222ca Jul 09 09:59:22.635984 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:22.635943 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 09:59:22.636071 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:22.636023 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 09:59:22.636071 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:22.636057 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 09:59:22.700691 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:22.700665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" event={"ID":"4257f47c-2113-4728-97c3-2089b6266d59","Type":"ContainerStarted","Data":"028e54cdc408496fc1140e9c0b835228a9c032ea4e6bbbeecdb25ad61d8222ca"} Jul 09 09:59:23.709380 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:23.709317 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" event={"ID":"c262fcfc-ee30-4f3a-8de5-e23a6c96f86b","Type":"ContainerStarted","Data":"624599bf490c6ce5dd63940322701f417e884458540cadf11b823d1cdfa17840"} Jul 09 09:59:23.764127 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:23.763103 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" podStartSLOduration=1.995993827 podStartE2EDuration="4.76308109s" totalImagesPullingTime="2.767087263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:23.762009851 +0000 UTC m=+301.960101649" watchObservedRunningTime="2026-07-09 09:59:23.76308109 +0000 UTC m=+301.961172890" Jul 09 09:59:24.700458 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:24.700422 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:24.705219 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:24.705196 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:24.714364 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:24.714339 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:24.715228 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:24.715205 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678vnp2r" Jul 09 09:59:24.747727 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:24.747697 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 09:59:24.747819 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:24.747758 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 09:59:25.720955 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:25.720917 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" event={"ID":"4257f47c-2113-4728-97c3-2089b6266d59","Type":"ContainerStarted","Data":"a021e32e646c07a2b3b5ee94abbaf0cc010a82c40b10e74514d58a5bc3e225a4"} Jul 09 09:59:25.721465 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:25.721162 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 09:59:25.722785 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:25.722759 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-kmbnl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 09:59:25.722894 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:25.722806 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" podUID="4257f47c-2113-4728-97c3-2089b6266d59" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 09:59:25.898581 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:25.898519 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" podStartSLOduration=2.774609324 podStartE2EDuration="4.898498166s" totalImagesPullingTime="2.123888842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:25.893951481 +0000 UTC m=+304.092043279" watchObservedRunningTime="2026-07-09 09:59:25.898498166 +0000 UTC m=+304.096589961" Jul 09 09:59:26.726018 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:26.725989 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-kmbnl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 09:59:26.726404 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:26.726036 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" podUID="4257f47c-2113-4728-97c3-2089b6266d59" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 09:59:27.084162 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:27.084059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:27.238855 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:27.238821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:27.382020 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:27.381938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:27.738235 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:27.738205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:28.152815 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:28.152735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:28.247097 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:28.247064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:28.501233 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:28.501198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 09:59:29.682249 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:29.682222 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-r7rvm" Jul 09 09:59:29.726223 ip-10-0-138-234 kubenswrapper[2576]: I0709 09:59:29.726194 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmbnl" Jul 09 10:00:09.858561 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:09.858529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:09.951964 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:09.951923 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:10.055361 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:10.055325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:39.066082 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:39.065982 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:42.120459 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:42.120416 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:44.472248 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:44.472213 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:45.903999 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:45.903970 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:46.145409 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:46.145380 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:47.452997 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:47.452952 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:49.326339 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:49.326301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:49.426645 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:49.426596 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:49.632493 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:49.632392 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:50.418938 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:50.418903 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:52.193842 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:52.193792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:54.154638 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:54.154609 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:54.262987 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:54.262955 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:54.460097 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:54.460016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:00:54.544774 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:00:54.544727 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:00.314531 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:00.314498 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:00.414987 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:00.414956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:00.525104 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:00.525073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:00.615240 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:00.615176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:03.200263 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:03.200226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:03.710315 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:03.710282 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:08.562552 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:08.562515 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:08.652791 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:08.652763 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:08.755436 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:08.755402 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:10.700848 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:10.700816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:10.799792 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:10.799760 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:11.505447 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:11.505405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:11.626602 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:11.626569 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:15.495577 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:15.495543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:15.560170 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:15.560123 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:23.496207 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:23.496146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:23.561838 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:23.561804 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:29.942710 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:29.942672 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:30.005032 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:30.004998 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:30.527250 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:30.527221 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:30.644981 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:30.644946 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:33.893349 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:33.893315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:33.993196 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:33.993139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:34.671822 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.671790 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h"] Jul 09 10:01:34.674431 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.674414 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.677522 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.677496 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mqsks\"" Jul 09 10:01:34.699568 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.699545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:34.745802 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.745775 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h"] Jul 09 10:01:34.781956 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.781928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782108 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.781959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782108 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.781984 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782108 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.782037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwgt2\" (UniqueName: \"kubernetes.io/projected/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-kube-api-access-kwgt2\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782108 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.782089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782367 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.782215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782367 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.782285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782367 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.782324 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.782491 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.782364 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.812325 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.812293 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:34.884181 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.883640 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884181 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.883704 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884181 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884053 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884181 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884093 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884181 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884123 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884181 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884181 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884593 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884218 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwgt2\" (UniqueName: \"kubernetes.io/projected/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-kube-api-access-kwgt2\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884593 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884261 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884593 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884748 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884687 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.884958 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.884936 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.885358 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.885332 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.885495 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.885367 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.885495 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.885392 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.886744 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.886722 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.886974 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.886954 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.904603 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.904575 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.905194 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.905144 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwgt2\" (UniqueName: \"kubernetes.io/projected/0454bc0d-d0dd-4dc1-833d-74ebf75a78ff-kube-api-access-kwgt2\") pod \"maas-default-gateway-openshift-default-5896d4dd49-x5x5h\" (UID: \"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:34.984496 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:34.984468 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:35.125299 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.125270 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h"] Jul 09 10:01:35.127130 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:01:35.127106 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0454bc0d_d0dd_4dc1_833d_74ebf75a78ff.slice/crio-52fcd0a904cbbe55a5eb961569d475036fc5aec9d13782f0d6f882501afc8b7f WatchSource:0}: Error finding container 52fcd0a904cbbe55a5eb961569d475036fc5aec9d13782f0d6f882501afc8b7f: Status 404 returned error can't find the container with id 52fcd0a904cbbe55a5eb961569d475036fc5aec9d13782f0d6f882501afc8b7f Jul 09 10:01:35.129140 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.129107 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:01:35.129235 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.129204 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:01:35.129281 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.129243 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:01:35.319890 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.319788 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" event={"ID":"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff","Type":"ContainerStarted","Data":"25dff4e26b2a5ef01b1cd7032f12d50e951e93e1c2ed097f2752de65e8aaa617"} Jul 09 10:01:35.319890 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.319841 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" event={"ID":"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff","Type":"ContainerStarted","Data":"52fcd0a904cbbe55a5eb961569d475036fc5aec9d13782f0d6f882501afc8b7f"} Jul 09 10:01:35.344616 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.344566 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" podStartSLOduration=1.344550969 podStartE2EDuration="1.344550969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:01:35.342860375 +0000 UTC m=+433.540952172" watchObservedRunningTime="2026-07-09 10:01:35.344550969 +0000 UTC m=+433.542642766" Jul 09 10:01:35.984929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:35.984895 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:36.355256 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.355145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:36.476339 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.476302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:36.541012 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.540982 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:01:36.543712 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.543697 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.546366 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.546346 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 10:01:36.546723 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.546705 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zvqhc\"" Jul 09 10:01:36.547569 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.547552 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 10:01:36.547670 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.547579 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 10:01:36.553797 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.553757 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:36.554260 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.554241 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:01:36.591832 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.591801 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:01:36.600395 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.600370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8717516-5e1a-4116-b297-7125c2893642-config-file\") pod \"limitador-limitador-54d6d988d5-zvh96\" (UID: \"b8717516-5e1a-4116-b297-7125c2893642\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.600506 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.600493 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwhqz\" (UniqueName: \"kubernetes.io/projected/b8717516-5e1a-4116-b297-7125c2893642-kube-api-access-pwhqz\") pod \"limitador-limitador-54d6d988d5-zvh96\" (UID: \"b8717516-5e1a-4116-b297-7125c2893642\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.701429 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.701365 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8717516-5e1a-4116-b297-7125c2893642-config-file\") pod \"limitador-limitador-54d6d988d5-zvh96\" (UID: \"b8717516-5e1a-4116-b297-7125c2893642\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.701559 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.701453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwhqz\" (UniqueName: \"kubernetes.io/projected/b8717516-5e1a-4116-b297-7125c2893642-kube-api-access-pwhqz\") pod \"limitador-limitador-54d6d988d5-zvh96\" (UID: \"b8717516-5e1a-4116-b297-7125c2893642\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.702026 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.702001 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8717516-5e1a-4116-b297-7125c2893642-config-file\") pod \"limitador-limitador-54d6d988d5-zvh96\" (UID: \"b8717516-5e1a-4116-b297-7125c2893642\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.711126 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.711104 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwhqz\" (UniqueName: \"kubernetes.io/projected/b8717516-5e1a-4116-b297-7125c2893642-kube-api-access-pwhqz\") pod \"limitador-limitador-54d6d988d5-zvh96\" (UID: \"b8717516-5e1a-4116-b297-7125c2893642\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.855354 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.855327 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:36.984299 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.984242 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:01:36.986538 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:01:36.986509 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8717516_5e1a_4116_b297_7125c2893642.slice/crio-8361f3af575ac0469df893f7473fab24356b674443e038157d28ee1467ff11a5 WatchSource:0}: Error finding container 8361f3af575ac0469df893f7473fab24356b674443e038157d28ee1467ff11a5: Status 404 returned error can't find the container with id 8361f3af575ac0469df893f7473fab24356b674443e038157d28ee1467ff11a5 Jul 09 10:01:36.988767 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.988751 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 10:01:36.990390 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:36.990369 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:37.258018 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:37.257936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:37.320427 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:37.320401 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:37.331232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:37.331210 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" event={"ID":"b8717516-5e1a-4116-b297-7125c2893642","Type":"ContainerStarted","Data":"8361f3af575ac0469df893f7473fab24356b674443e038157d28ee1467ff11a5"} Jul 09 10:01:37.331514 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:37.331494 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:37.332338 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:37.332323 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:01:40.347921 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:40.347886 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" event={"ID":"b8717516-5e1a-4116-b297-7125c2893642","Type":"ContainerStarted","Data":"fb57a9d047bd58b32cd2fd81546e70edb8cf67b057def6f0c1945b66f493df61"} Jul 09 10:01:40.348342 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:40.348025 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:40.371506 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:40.371457 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" podStartSLOduration=1.555790425 podStartE2EDuration="4.371442636s" totalImagesPullingTime="2.815652211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:01:40.370532596 +0000 UTC m=+438.568624405" watchObservedRunningTime="2026-07-09 10:01:40.371442636 +0000 UTC m=+438.569534433" Jul 09 10:01:51.353671 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:51.353592 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zvh96" Jul 09 10:01:57.579630 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:57.579583 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:57.674280 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:57.674248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:01:58.059336 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:01:58.059307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:10.846196 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:10.846147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:11.278502 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:11.278473 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:18.051970 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:18.051933 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:18.149537 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:18.149505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:18.249373 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:18.249338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:21.743814 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:21.743781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:21.844486 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:21.844461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:51.347520 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:51.347484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:51.811268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:51.811233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:52.283077 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:52.283050 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:52.383163 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:52.383110 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:52.472353 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:52.472318 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:02:54.472566 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:54.472532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:02:59.711273 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:59.711248 2576 generic.go:356] "Generic (PLEG): container finished" podID="0454bc0d-d0dd-4dc1-833d-74ebf75a78ff" containerID="25dff4e26b2a5ef01b1cd7032f12d50e951e93e1c2ed097f2752de65e8aaa617" exitCode=0 Jul 09 10:02:59.711581 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:59.711321 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" event={"ID":"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff","Type":"ContainerDied","Data":"25dff4e26b2a5ef01b1cd7032f12d50e951e93e1c2ed097f2752de65e8aaa617"} Jul 09 10:02:59.711703 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:59.711689 2576 scope.go:117] "RemoveContainer" containerID="25dff4e26b2a5ef01b1cd7032f12d50e951e93e1c2ed097f2752de65e8aaa617" Jul 09 10:02:59.712744 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:59.712715 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:02:59.712813 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:59.712784 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:02:59.712848 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:02:59.712838 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:03:00.718227 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:00.718186 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" event={"ID":"0454bc0d-d0dd-4dc1-833d-74ebf75a78ff","Type":"ContainerStarted","Data":"5e965ddc1ef6651984c672c2922719bba5e5a7f072c31d508e688a8864d193ec"} Jul 09 10:03:00.985073 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:00.984979 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:03:00.985288 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:00.985264 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-x5x5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 09 10:03:00.985335 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:00.985315 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" podUID="0454bc0d-d0dd-4dc1-833d-74ebf75a78ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 09 10:03:01.985377 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:01.985345 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-x5x5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 09 10:03:01.985718 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:01.985410 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" podUID="0454bc0d-d0dd-4dc1-833d-74ebf75a78ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 09 10:03:02.843197 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:02.843164 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:02.943921 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:02.943882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:02.985742 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:02.985709 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-x5x5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 09 10:03:02.986080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:02.985760 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" podUID="0454bc0d-d0dd-4dc1-833d-74ebf75a78ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 09 10:03:03.985770 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:03.985737 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-x5x5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 09 10:03:03.986180 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:03.985790 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" podUID="0454bc0d-d0dd-4dc1-833d-74ebf75a78ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 09 10:03:04.070476 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:04.070447 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:04.988763 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:04.988737 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:03:04.989124 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:04.989007 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:03:04.989625 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:04.989609 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-x5x5h" Jul 09 10:03:16.584023 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:16.583943 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:03:17.977420 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:17.977380 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:21.425440 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:21.425405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:22.211436 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:22.211400 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:03:23.250706 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:23.250669 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:32.407464 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:32.407426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:33.593889 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:33.593857 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:03:35.736205 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:35.736169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:37.984758 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:37.984723 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:03:40.450838 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:40.450805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:42.464142 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:42.464109 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:44.503166 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.503121 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck"] Jul 09 10:03:44.505922 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.505903 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.510709 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.510688 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 10:03:44.510806 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.510763 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 10:03:44.511078 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.511056 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 10:03:44.511931 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.511911 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xzxkm\"" Jul 09 10:03:44.527735 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.527713 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck"] Jul 09 10:03:44.542589 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.542560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:44.546530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.546510 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.546604 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.546551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hpc2\" (UniqueName: \"kubernetes.io/projected/6db2af6f-f172-4807-b20f-999eb4f96509-kube-api-access-5hpc2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.546604 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.546572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.546712 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.546696 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.546750 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.546723 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.546750 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.546744 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6db2af6f-f172-4807-b20f-999eb4f96509-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.647432 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647403 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.647600 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hpc2\" (UniqueName: \"kubernetes.io/projected/6db2af6f-f172-4807-b20f-999eb4f96509-kube-api-access-5hpc2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.647600 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647502 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.647751 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.647888 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.647958 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6db2af6f-f172-4807-b20f-999eb4f96509-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.648021 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.647986 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.648021 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.648012 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.648129 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.648111 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.649856 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.649827 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6db2af6f-f172-4807-b20f-999eb4f96509-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.650114 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.650098 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6db2af6f-f172-4807-b20f-999eb4f96509-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.669748 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.669729 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hpc2\" (UniqueName: \"kubernetes.io/projected/6db2af6f-f172-4807-b20f-999eb4f96509-kube-api-access-5hpc2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-25rck\" (UID: \"6db2af6f-f172-4807-b20f-999eb4f96509\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.816862 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.816803 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:44.961295 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:44.961273 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck"] Jul 09 10:03:44.963914 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:03:44.963892 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db2af6f_f172_4807_b20f_999eb4f96509.slice/crio-727dbc7d65dfd3cb43f2509ce761f3c37ff33549ac8c6eaa4c3d25e536eb3e54 WatchSource:0}: Error finding container 727dbc7d65dfd3cb43f2509ce761f3c37ff33549ac8c6eaa4c3d25e536eb3e54: Status 404 returned error can't find the container with id 727dbc7d65dfd3cb43f2509ce761f3c37ff33549ac8c6eaa4c3d25e536eb3e54 Jul 09 10:03:45.934692 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:45.934656 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" event={"ID":"6db2af6f-f172-4807-b20f-999eb4f96509","Type":"ContainerStarted","Data":"727dbc7d65dfd3cb43f2509ce761f3c37ff33549ac8c6eaa4c3d25e536eb3e54"} Jul 09 10:03:46.909931 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:46.909894 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:03:46.940874 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:46.940842 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" event={"ID":"6db2af6f-f172-4807-b20f-999eb4f96509","Type":"ContainerStarted","Data":"d5d50d06e2e67acb29bbe6a803cf16c08dcd6e7d0b20327badee2e6e1bf14c83"} Jul 09 10:03:46.967128 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:46.967081 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" podStartSLOduration=1.883034157 podStartE2EDuration="2.967066471s" totalImagesPullingTime="1.084032314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:03:46.964384646 +0000 UTC m=+565.162476443" watchObservedRunningTime="2026-07-09 10:03:46.967066471 +0000 UTC m=+565.165158268" Jul 09 10:03:51.413527 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:51.413494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:03:53.900479 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:53.900449 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:03:54.817882 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:54.817847 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:54.818057 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:54.817916 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:54.830233 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:54.830211 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:03:54.988249 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:03:54.988227 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-25rck" Jul 09 10:04:38.721437 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:04:38.721398 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:04:40.870470 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:04:40.870431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:05:14.742914 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:14.742884 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:05:14.840989 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:14.840953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:05:21.864399 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:21.864358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:05:26.497255 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:26.497218 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:05:36.988592 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:36.988560 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:05:48.382270 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:48.382238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:05:53.805745 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:53.805708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:05:53.907435 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:53.907406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:05:56.994624 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:05:56.994588 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:06:07.689564 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:07.689532 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:06:10.006748 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:10.006707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:11.603733 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:11.603700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:47.372951 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:47.372870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:47.462744 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:47.462704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:47.582460 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:47.582422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:54.145878 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:54.145838 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:55.029277 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:55.029240 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:06:55.092309 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:06:55.092274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:07:01.702116 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:01.702075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:07:01.803172 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:01.803101 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:07:02.014363 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:02.014322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:07:02.149212 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:02.149168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:07:17.797001 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:17.796967 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:07:28.090632 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:28.090601 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:07:36.613360 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:36.613331 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:07:43.965625 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:43.965588 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:07:45.687313 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:45.687277 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:07:54.884771 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:07:54.884735 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:08:06.686977 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:08:06.686941 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:08:17.483273 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:08:17.483241 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:08:27.688432 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:08:27.688396 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:08:49.615504 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:08:49.615470 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:08:59.336530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:08:59.336490 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:09:00.220124 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:09:00.220080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:09:03.191552 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:09:03.191516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:09:22.696623 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:09:22.696547 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:09:33.099490 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:09:33.099448 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:09:50.089656 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:09:50.089624 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:10:00.203572 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:10:00.203540 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:10:10.190733 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:10:10.190697 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:10:19.493403 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:10:19.493365 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:10:28.394432 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:10:28.394397 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:12:18.599908 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:12:18.599835 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:12:27.587832 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:12:27.587793 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:12:42.688442 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:12:42.688410 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:12:55.002711 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:12:55.002676 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:13:06.810006 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:06.809974 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:13:15.793386 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:15.793357 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:13:25.692385 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:25.692351 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:13:36.293742 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:36.293709 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:13:39.593230 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:39.593197 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:13:40.794767 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:40.794725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:13:42.814806 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:42.814774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:13:46.589460 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:46.589387 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:13:57.209441 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:13:57.209403 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:14:08.687652 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:14:08.687623 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:14:36.791438 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:14:36.791403 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:14:45.890815 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:14:45.890779 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:14:56.289235 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:14:56.289194 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:15:06.393445 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:06.393414 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:15:13.089882 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:13.089840 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:15:29.001754 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:29.001666 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:15:40.027987 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:40.027954 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:15:45.728804 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:45.728764 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:15:49.404623 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:49.404589 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:15:59.713654 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:15:59.713620 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:16:09.317389 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:09.317353 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:16:18.896971 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:18.896934 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:16:27.497886 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:27.497856 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:16:27.595445 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:27.595412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:16:32.099324 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:32.099288 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:16:41.002112 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:41.002077 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:16:51.091778 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:16:51.091695 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:00.693201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:00.693166 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:12.792549 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:12.792518 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:20.624705 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:20.624672 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:31.924213 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:31.924172 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:44.686860 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:44.686823 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:54.984213 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:54.984169 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:17:59.767176 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:59.767120 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:17:59.836261 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:17:59.836228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:00.242621 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:00.242587 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:06.556655 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.556624 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:06.664210 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.664179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:06.756062 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.756036 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp"] Jul 09 10:18:06.759577 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.759560 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.762932 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.762912 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-848aa2ca-openshift-default-dockercfg-lsztz\"" Jul 09 10:18:06.792740 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.792715 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp"] Jul 09 10:18:06.806706 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.806606 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:06.839825 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-envoy\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.839825 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-credential-socket\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839855 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b911a94-3297-4903-8824-30d8b71cf808-istio-podinfo\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839892 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-istio-token\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839934 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-data\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drjtt\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-kube-api-access-drjtt\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.839996 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b911a94-3297-4903-8824-30d8b71cf808-istiod-ca-cert\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.840042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-certs\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.847687 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.840084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-socket\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.932851 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.932818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:06.941041 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941013 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-socket\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941061 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-envoy\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941116 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-credential-socket\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941147 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b911a94-3297-4903-8824-30d8b71cf808-istio-podinfo\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941187 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-istio-token\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941407 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-data\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941407 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941256 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drjtt\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-kube-api-access-drjtt\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941407 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b911a94-3297-4903-8824-30d8b71cf808-istiod-ca-cert\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941618 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-certs\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941618 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941430 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-socket\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941618 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941464 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-credential-socket\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941618 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941563 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-data\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941758 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941697 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-certs\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.941980 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.941947 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b911a94-3297-4903-8824-30d8b71cf808-istiod-ca-cert\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.943572 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.943546 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-envoy\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.943683 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.943669 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b911a94-3297-4903-8824-30d8b71cf808-istio-podinfo\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.955480 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.955458 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-istio-token\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:06.962908 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:06.962884 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drjtt\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-kube-api-access-drjtt\") pod \"e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:07.069313 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:07.069188 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:07.202230 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:07.202207 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp"] Jul 09 10:18:07.204311 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:18:07.204285 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b911a94_3297_4903_8824_30d8b71cf808.slice/crio-b75445f6621307891bc53c100002345908e5675ea0c7b2763e10f0d21e4c7a1d WatchSource:0}: Error finding container b75445f6621307891bc53c100002345908e5675ea0c7b2763e10f0d21e4c7a1d: Status 404 returned error can't find the container with id b75445f6621307891bc53c100002345908e5675ea0c7b2763e10f0d21e4c7a1d Jul 09 10:18:07.206338 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:07.206305 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:07.206423 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:07.206368 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:07.206423 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:07.206398 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:07.991638 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:07.991600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:08.066143 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.066111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" event={"ID":"6b911a94-3297-4903-8824-30d8b71cf808","Type":"ContainerStarted","Data":"c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b"} Jul 09 10:18:08.066346 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.066185 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" event={"ID":"6b911a94-3297-4903-8824-30d8b71cf808","Type":"ContainerStarted","Data":"b75445f6621307891bc53c100002345908e5675ea0c7b2763e10f0d21e4c7a1d"} Jul 09 10:18:08.069758 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.069726 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:08.074535 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.074515 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:08.078198 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.078176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:08.094212 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.094169 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" podStartSLOduration=2.094139247 podStartE2EDuration="2.094139247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:18:08.092478981 +0000 UTC m=+1426.290570781" watchObservedRunningTime="2026-07-09 10:18:08.094139247 +0000 UTC m=+1426.292231108" Jul 09 10:18:08.150001 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.149976 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:08.334707 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.334619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:08.804572 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.804538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:08.900717 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:08.900686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:09.071461 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:09.071381 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:09.072328 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:09.072308 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:10.693622 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:10.693575 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:18:10.851865 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:10.851826 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:13.282979 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:13.282948 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:13.431648 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:13.431616 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp"] Jul 09 10:18:13.431818 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:13.431786 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" podUID="6b911a94-3297-4903-8824-30d8b71cf808" containerName="istio-proxy" containerID="cri-o://c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b" gracePeriod=30 Jul 09 10:18:13.455091 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:13.455054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:13.624025 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:13.623945 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:13.818175 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:13.818121 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:14.834082 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:14.834050 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:18.880746 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.880720 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:18.956062 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.955994 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-envoy\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956062 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956024 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b911a94-3297-4903-8824-30d8b71cf808-istio-podinfo\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956062 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956047 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b911a94-3297-4903-8824-30d8b71cf808-istiod-ca-cert\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956085 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drjtt\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-kube-api-access-drjtt\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956117 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-credential-socket\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956142 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-istio-token\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956219 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-data\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956278 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-certs\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956311 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-socket\") pod \"6b911a94-3297-4903-8824-30d8b71cf808\" (UID: \"6b911a94-3297-4903-8824-30d8b71cf808\") " Jul 09 10:18:18.956673 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956493 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:18.956735 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956672 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:18.956797 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956771 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:18.956846 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956792 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:18.956901 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.956855 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:18.957212 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.957186 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b911a94-3297-4903-8824-30d8b71cf808-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:18:18.957307 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.957211 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:18.958394 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.958365 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:18:18.958501 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.958420 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b911a94-3297-4903-8824-30d8b71cf808-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:18:18.958501 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.958421 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:18.958501 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:18.958468 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-kube-api-access-drjtt" (OuterVolumeSpecName: "kube-api-access-drjtt") pod "6b911a94-3297-4903-8824-30d8b71cf808" (UID: "6b911a94-3297-4903-8824-30d8b71cf808"). InnerVolumeSpecName "kube-api-access-drjtt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:18:19.057322 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057287 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.057322 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057318 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b911a94-3297-4903-8824-30d8b71cf808-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.057509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057332 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b911a94-3297-4903-8824-30d8b71cf808-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.057509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057345 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drjtt\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-kube-api-access-drjtt\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.057509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057357 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b911a94-3297-4903-8824-30d8b71cf808-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.057509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057368 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.057509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.057378 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b911a94-3297-4903-8824-30d8b71cf808-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:19.124997 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.124969 2576 generic.go:356] "Generic (PLEG): container finished" podID="6b911a94-3297-4903-8824-30d8b71cf808" containerID="c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b" exitCode=0 Jul 09 10:18:19.125205 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.125014 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" Jul 09 10:18:19.125205 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.125031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" event={"ID":"6b911a94-3297-4903-8824-30d8b71cf808","Type":"ContainerDied","Data":"c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b"} Jul 09 10:18:19.125205 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.125082 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp" event={"ID":"6b911a94-3297-4903-8824-30d8b71cf808","Type":"ContainerDied","Data":"b75445f6621307891bc53c100002345908e5675ea0c7b2763e10f0d21e4c7a1d"} Jul 09 10:18:19.125205 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.125110 2576 scope.go:117] "RemoveContainer" containerID="c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b" Jul 09 10:18:19.134232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.134215 2576 scope.go:117] "RemoveContainer" containerID="c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b" Jul 09 10:18:19.134455 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:18:19.134438 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b\": container with ID starting with c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b not found: ID does not exist" containerID="c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b" Jul 09 10:18:19.134501 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.134463 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b"} err="failed to get container status \"c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b\": rpc error: code = NotFound desc = could not find container \"c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b\": container with ID starting with c7c5e7c77a3c52e68f4a57f965ded8bc56e2a808738a7bff562712f074c6b99b not found: ID does not exist" Jul 09 10:18:19.148361 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.148339 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp"] Jul 09 10:18:19.152865 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.152843 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-848aa2ca-openshift-default-c9fb984f9-8j9wp"] Jul 09 10:18:19.890595 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.890563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:19.962418 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:19.962385 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:20.076352 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.076318 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb"] Jul 09 10:18:20.076806 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.076787 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b911a94-3297-4903-8824-30d8b71cf808" containerName="istio-proxy" Jul 09 10:18:20.076806 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.076807 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b911a94-3297-4903-8824-30d8b71cf808" containerName="istio-proxy" Jul 09 10:18:20.077002 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.076919 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b911a94-3297-4903-8824-30d8b71cf808" containerName="istio-proxy" Jul 09 10:18:20.080489 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.080469 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.084340 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.084317 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ba8169ec-openshift-default-dockercfg-cm7tx\"" Jul 09 10:18:20.096377 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.096354 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb"] Jul 09 10:18:20.147710 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.147646 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:20.166980 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.166954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-token\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167089 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.166998 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-certs\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167089 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-podinfo\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167089 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167040 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqkct\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-kube-api-access-kqkct\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167089 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167066 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-data\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167089 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167087 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-credential-socket\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167303 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-envoy\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167303 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-socket\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.167303 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.167208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0e9c6a-6295-46ac-b614-7053b225bd30-istiod-ca-cert\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.206192 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.206161 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:18:20.215412 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.215388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:20.268587 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268563 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-token\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268684 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-certs\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268684 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-podinfo\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268684 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268645 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqkct\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-kube-api-access-kqkct\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268684 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-data\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268890 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-credential-socket\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268890 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-envoy\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268890 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268797 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-socket\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.268890 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.268839 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0e9c6a-6295-46ac-b614-7053b225bd30-istiod-ca-cert\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.269124 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.269096 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-data\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.269206 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.269126 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-credential-socket\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.269315 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.269281 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-socket\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.269667 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.269645 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0e9c6a-6295-46ac-b614-7053b225bd30-istiod-ca-cert\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.269777 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.269761 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-certs\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.271067 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.271046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-envoy\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.271141 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.271114 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-podinfo\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.277951 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.277923 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-token\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.278231 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.278211 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqkct\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-kube-api-access-kqkct\") pod \"e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.329943 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.329905 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b911a94-3297-4903-8824-30d8b71cf808" path="/var/lib/kubelet/pods/6b911a94-3297-4903-8824-30d8b71cf808/volumes" Jul 09 10:18:20.391794 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.391764 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:20.528920 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.528899 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb"] Jul 09 10:18:20.530638 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:18:20.530614 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0e9c6a_6295_46ac_b614_7053b225bd30.slice/crio-30ef0a7603fc662a60a476f64ee0d5e90c5996eecfc68ddff179b2432e0f1294 WatchSource:0}: Error finding container 30ef0a7603fc662a60a476f64ee0d5e90c5996eecfc68ddff179b2432e0f1294: Status 404 returned error can't find the container with id 30ef0a7603fc662a60a476f64ee0d5e90c5996eecfc68ddff179b2432e0f1294 Jul 09 10:18:20.532747 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.532715 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:20.532805 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.532780 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:20.532837 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:20.532808 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:21.138642 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.138611 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" event={"ID":"5a0e9c6a-6295-46ac-b614-7053b225bd30","Type":"ContainerStarted","Data":"0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d"} Jul 09 10:18:21.139024 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.138669 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" event={"ID":"5a0e9c6a-6295-46ac-b614-7053b225bd30","Type":"ContainerStarted","Data":"30ef0a7603fc662a60a476f64ee0d5e90c5996eecfc68ddff179b2432e0f1294"} Jul 09 10:18:21.163972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.163918 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" podStartSLOduration=1.163903174 podStartE2EDuration="1.163903174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:18:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:18:21.161848229 +0000 UTC m=+1439.359940027" watchObservedRunningTime="2026-07-09 10:18:21.163903174 +0000 UTC m=+1439.361994971" Jul 09 10:18:21.355084 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.355055 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:21.392557 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.392486 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:21.396916 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.396893 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:21.438444 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.438417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:21.513167 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.513136 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:21.959855 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:21.959821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:22.143548 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:22.143521 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:22.144509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:22.144490 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:22.216784 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:22.216751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:22.315129 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:22.315103 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:23.500956 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:23.500922 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:26.470214 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:26.470176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:26.896570 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:26.896486 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:32.602492 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.602459 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb"] Jul 09 10:18:32.603906 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.602724 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" podUID="5a0e9c6a-6295-46ac-b614-7053b225bd30" containerName="istio-proxy" containerID="cri-o://0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d" gracePeriod=30 Jul 09 10:18:32.627718 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.627693 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:32.759080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.759048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:32.861519 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.861465 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:32.907534 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907508 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-socket\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907713 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907546 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-credential-socket\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907713 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907573 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0e9c6a-6295-46ac-b614-7053b225bd30-istiod-ca-cert\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907713 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907642 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-certs\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907713 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907664 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-data\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907713 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907704 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqkct\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-kube-api-access-kqkct\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907727 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-envoy\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907762 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-podinfo\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.907972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907802 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:32.907972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907816 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:32.907972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907832 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-token\") pod \"5a0e9c6a-6295-46ac-b614-7053b225bd30\" (UID: \"5a0e9c6a-6295-46ac-b614-7053b225bd30\") " Jul 09 10:18:32.908253 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.907993 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:32.908253 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.908007 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0e9c6a-6295-46ac-b614-7053b225bd30-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:18:32.908253 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.908230 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:32.908253 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.908250 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:32.908470 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.908264 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0e9c6a-6295-46ac-b614-7053b225bd30-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:32.908470 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.908278 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:32.909073 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.909039 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:32.909924 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.909898 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:32.910251 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.910231 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:18:32.910416 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.910390 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-kube-api-access-kqkct" (OuterVolumeSpecName: "kube-api-access-kqkct") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "kube-api-access-kqkct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:18:32.911037 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:32.911012 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a0e9c6a-6295-46ac-b614-7053b225bd30" (UID: "5a0e9c6a-6295-46ac-b614-7053b225bd30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:18:33.008859 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.008830 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:33.008859 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.008858 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqkct\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-kube-api-access-kqkct\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:33.009013 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.008868 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:33.009013 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.008877 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:33.009013 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.008884 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0e9c6a-6295-46ac-b614-7053b225bd30-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:33.206662 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.206575 2576 generic.go:356] "Generic (PLEG): container finished" podID="5a0e9c6a-6295-46ac-b614-7053b225bd30" containerID="0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d" exitCode=0 Jul 09 10:18:33.206662 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.206621 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" Jul 09 10:18:33.206844 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.206660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" event={"ID":"5a0e9c6a-6295-46ac-b614-7053b225bd30","Type":"ContainerDied","Data":"0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d"} Jul 09 10:18:33.206844 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.206714 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb" event={"ID":"5a0e9c6a-6295-46ac-b614-7053b225bd30","Type":"ContainerDied","Data":"30ef0a7603fc662a60a476f64ee0d5e90c5996eecfc68ddff179b2432e0f1294"} Jul 09 10:18:33.206844 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.206739 2576 scope.go:117] "RemoveContainer" containerID="0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d" Jul 09 10:18:33.216307 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.216292 2576 scope.go:117] "RemoveContainer" containerID="0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d" Jul 09 10:18:33.216535 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:18:33.216515 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d\": container with ID starting with 0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d not found: ID does not exist" containerID="0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d" Jul 09 10:18:33.216612 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.216542 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d"} err="failed to get container status \"0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d\": rpc error: code = NotFound desc = could not find container \"0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d\": container with ID starting with 0269bc5a53805708213468ffdf70c3887d6484d9864677f5fdc96a8880d2f91d not found: ID does not exist" Jul 09 10:18:33.231406 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.231379 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb"] Jul 09 10:18:33.237355 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:33.237336 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ba8169ec-openshift-default-5d76f5bfb5-x48gb"] Jul 09 10:18:34.330928 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:34.330895 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a0e9c6a-6295-46ac-b614-7053b225bd30" path="/var/lib/kubelet/pods/5a0e9c6a-6295-46ac-b614-7053b225bd30/volumes" Jul 09 10:18:39.748801 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:39.748769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:39.848841 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:39.848813 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:39.998576 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:39.998550 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m"] Jul 09 10:18:39.999043 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:39.999004 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a0e9c6a-6295-46ac-b614-7053b225bd30" containerName="istio-proxy" Jul 09 10:18:39.999043 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:39.999021 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a0e9c6a-6295-46ac-b614-7053b225bd30" containerName="istio-proxy" Jul 09 10:18:39.999124 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:39.999095 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a0e9c6a-6295-46ac-b614-7053b225bd30" containerName="istio-proxy" Jul 09 10:18:40.002042 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.002028 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.005603 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.005584 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-92153720-openshift-default-dockercfg-8gs74\"" Jul 09 10:18:40.032560 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.032533 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m"] Jul 09 10:18:40.073771 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.073747 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx5z7\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-kube-api-access-bx5z7\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.073883 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.073781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-credential-socket\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.073883 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.073799 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-socket\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.073965 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.073887 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-data\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.074007 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.073960 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-token\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.074007 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.073990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-envoy\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.074083 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.074037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-certs\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.074083 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.074060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c16960-5a3a-4040-b658-b39fb1eadc1c-istiod-ca-cert\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.074209 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.074127 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-podinfo\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.081209 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.081188 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:40.175290 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175259 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-data\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175446 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-token\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175446 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175353 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-envoy\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175446 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175406 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-certs\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175446 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c16960-5a3a-4040-b658-b39fb1eadc1c-istiod-ca-cert\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175665 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-podinfo\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175665 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx5z7\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-kube-api-access-bx5z7\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175665 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175560 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-credential-socket\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175665 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175589 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-socket\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175981 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175950 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-data\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.175981 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.175972 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-credential-socket\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.176175 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.176026 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-certs\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.176175 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.176082 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-socket\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.176398 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.176378 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c16960-5a3a-4040-b658-b39fb1eadc1c-istiod-ca-cert\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.177657 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.177636 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-envoy\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.177897 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.177877 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-podinfo\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.191897 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.191871 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-token\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.196896 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.196873 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx5z7\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-kube-api-access-bx5z7\") pod \"e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.313389 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.313337 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:40.445398 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.445375 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m"] Jul 09 10:18:40.449433 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:18:40.449251 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c16960_5a3a_4040_b658_b39fb1eadc1c.slice/crio-b830cf48dc1f5de030cf1d6648819c02dcb014ff5a5e4a58b2ac06b8994d584c WatchSource:0}: Error finding container b830cf48dc1f5de030cf1d6648819c02dcb014ff5a5e4a58b2ac06b8994d584c: Status 404 returned error can't find the container with id b830cf48dc1f5de030cf1d6648819c02dcb014ff5a5e4a58b2ac06b8994d584c Jul 09 10:18:40.451204 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.451174 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:40.451280 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.451243 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:40.451280 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:40.451274 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:18:41.010857 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:41.010743 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:41.247313 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:41.247276 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" event={"ID":"95c16960-5a3a-4040-b658-b39fb1eadc1c","Type":"ContainerStarted","Data":"a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e"} Jul 09 10:18:41.247475 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:41.247322 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" event={"ID":"95c16960-5a3a-4040-b658-b39fb1eadc1c","Type":"ContainerStarted","Data":"b830cf48dc1f5de030cf1d6648819c02dcb014ff5a5e4a58b2ac06b8994d584c"} Jul 09 10:18:41.276587 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:41.276476 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" podStartSLOduration=2.276451861 podStartE2EDuration="2.276451861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:18:41.27105887 +0000 UTC m=+1459.469150679" watchObservedRunningTime="2026-07-09 10:18:41.276451861 +0000 UTC m=+1459.474543659" Jul 09 10:18:41.313895 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:41.313861 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:41.318751 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:41.318727 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:42.252146 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:42.252115 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:42.253084 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:42.253064 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:43.792736 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:43.792694 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:18:44.898784 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:44.898750 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:45.657606 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:45.657567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:45.809562 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:45.809528 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m"] Jul 09 10:18:45.809777 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:45.809749 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" podUID="95c16960-5a3a-4040-b658-b39fb1eadc1c" containerName="istio-proxy" containerID="cri-o://a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e" gracePeriod=30 Jul 09 10:18:45.841140 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:45.841112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:45.945818 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:45.945695 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:46.085510 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:46.085441 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:48.185707 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:48.185667 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:50.673357 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:50.673328 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:51.755530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.755483 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:51.898004 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.897871 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-certs\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898004 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.897914 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx5z7\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-kube-api-access-bx5z7\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898004 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.897948 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-envoy\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898004 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.897984 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-token\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898372 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898024 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-podinfo\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898372 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898053 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-credential-socket\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898372 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-data\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898372 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898117 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c16960-5a3a-4040-b658-b39fb1eadc1c-istiod-ca-cert\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898372 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898220 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-socket\") pod \"95c16960-5a3a-4040-b658-b39fb1eadc1c\" (UID: \"95c16960-5a3a-4040-b658-b39fb1eadc1c\") " Jul 09 10:18:51.898776 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898750 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:51.898953 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898798 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:51.898953 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.898831 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:51.899061 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.899009 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95c16960-5a3a-4040-b658-b39fb1eadc1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:18:51.899671 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.899643 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:51.900516 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.900484 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-kube-api-access-bx5z7" (OuterVolumeSpecName: "kube-api-access-bx5z7") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "kube-api-access-bx5z7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:18:51.900615 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.900580 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:18:51.900718 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.900697 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:18:51.900795 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:51.900780 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "95c16960-5a3a-4040-b658-b39fb1eadc1c" (UID: "95c16960-5a3a-4040-b658-b39fb1eadc1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:18:52.000038 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000009 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000038 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000034 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx5z7\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-kube-api-access-bx5z7\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000051 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000064 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000074 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000085 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000096 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000107 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c16960-5a3a-4040-b658-b39fb1eadc1c-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.000242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.000119 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c16960-5a3a-4040-b658-b39fb1eadc1c-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:18:52.276911 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.276882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:52.306083 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.306055 2576 generic.go:356] "Generic (PLEG): container finished" podID="95c16960-5a3a-4040-b658-b39fb1eadc1c" containerID="a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e" exitCode=0 Jul 09 10:18:52.306265 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.306098 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" Jul 09 10:18:52.306265 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.306115 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" event={"ID":"95c16960-5a3a-4040-b658-b39fb1eadc1c","Type":"ContainerDied","Data":"a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e"} Jul 09 10:18:52.306265 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.306195 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m" event={"ID":"95c16960-5a3a-4040-b658-b39fb1eadc1c","Type":"ContainerDied","Data":"b830cf48dc1f5de030cf1d6648819c02dcb014ff5a5e4a58b2ac06b8994d584c"} Jul 09 10:18:52.306265 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.306231 2576 scope.go:117] "RemoveContainer" containerID="a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e" Jul 09 10:18:52.315738 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.315722 2576 scope.go:117] "RemoveContainer" containerID="a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e" Jul 09 10:18:52.316001 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:18:52.315984 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e\": container with ID starting with a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e not found: ID does not exist" containerID="a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e" Jul 09 10:18:52.316048 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.316009 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e"} err="failed to get container status \"a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e\": rpc error: code = NotFound desc = could not find container \"a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e\": container with ID starting with a87590a4fc74ed84ea658fa6f7ac069f055d3e6288948c0230c1759fa0d3416e not found: ID does not exist" Jul 09 10:18:52.330898 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.330877 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m"] Jul 09 10:18:52.335309 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.335292 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-92153720-openshift-default-5948ffbb59-jtm9m"] Jul 09 10:18:52.343441 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.343417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:52.803576 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:52.803544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:53.098515 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:53.098436 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:18:53.227477 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:53.227443 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:53.447497 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:53.447413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:18:54.329405 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:54.329376 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95c16960-5a3a-4040-b658-b39fb1eadc1c" path="/var/lib/kubelet/pods/95c16960-5a3a-4040-b658-b39fb1eadc1c/volumes" Jul 09 10:18:54.600036 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:18:54.599957 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:01.965419 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:01.965384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:02.066694 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:02.066661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:27.739267 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:27.739232 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:34.016215 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:34.016180 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:34.125944 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:34.125915 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:34.216593 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:34.216565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:50.184634 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:50.184554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:56.666813 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:56.666775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:56.770301 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:56.770274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:19:58.082078 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:19:58.082046 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:07.231033 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:07.230998 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:08.076074 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:08.076036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:14.592166 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:14.592128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:14.692399 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:14.692369 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:14.792649 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:14.792615 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:14.892233 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:14.892132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:15.741526 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:15.741495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:15.842119 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:15.842088 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:15.935766 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:15.935732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:16.814096 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:16.814062 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:20:17.751209 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:17.751146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:24.540426 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:24.540397 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:20:25.771642 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:25.771611 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:25.873747 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:25.873719 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:26.574615 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:26.574574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:31.797816 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:31.797774 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:20:32.810064 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:32.810036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:32.873118 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:32.873083 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:33.090866 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.090778 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:33.433922 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.433846 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr"] Jul 09 10:20:33.434264 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.434251 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c16960-5a3a-4040-b658-b39fb1eadc1c" containerName="istio-proxy" Jul 09 10:20:33.434315 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.434267 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c16960-5a3a-4040-b658-b39fb1eadc1c" containerName="istio-proxy" Jul 09 10:20:33.434353 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.434333 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c16960-5a3a-4040-b658-b39fb1eadc1c" containerName="istio-proxy" Jul 09 10:20:33.436324 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.436309 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.441807 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.441777 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1df781d2-openshift-default-dockercfg-kxsg6\"" Jul 09 10:20:33.451069 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.451047 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:33.463463 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.463440 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr"] Jul 09 10:20:33.601814 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.601779 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-envoy\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.601814 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.601818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istiod-ca-cert\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602019 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.601865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s982h\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-kube-api-access-s982h\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602019 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.601888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-socket\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602019 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.601909 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-token\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602019 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.601965 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-data\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602019 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.602012 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-podinfo\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602206 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.602049 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-credential-socket\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.602206 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.602132 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-certs\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.612532 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.612507 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:33.703344 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703264 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-token\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703344 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703297 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-data\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703344 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703330 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-podinfo\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703571 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-credential-socket\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703571 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703444 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-certs\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703571 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703526 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-envoy\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703571 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istiod-ca-cert\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703778 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703595 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s982h\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-kube-api-access-s982h\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703778 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-socket\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.703778 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.703726 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-data\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.704060 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.704039 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-credential-socket\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.704144 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.704123 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-certs\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.704232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.704207 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-socket\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.704513 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.704488 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istiod-ca-cert\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.705810 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.705793 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-podinfo\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.705887 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.705825 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-envoy\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.717927 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.717907 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s982h\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-kube-api-access-s982h\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.719097 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.719077 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-token\") pod \"e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.747116 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.747092 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:33.901779 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.901753 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr"] Jul 09 10:20:33.902165 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:20:33.902131 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb360ea4_948b_4c98_a0c2_edb02c7a8255.slice/crio-c86db9ebb27848cd22bcba66159646aef7280f61a5991cd1d4e12e3e450cf913 WatchSource:0}: Error finding container c86db9ebb27848cd22bcba66159646aef7280f61a5991cd1d4e12e3e450cf913: Status 404 returned error can't find the container with id c86db9ebb27848cd22bcba66159646aef7280f61a5991cd1d4e12e3e450cf913 Jul 09 10:20:33.904267 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.904231 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:20:33.904345 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.904299 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:20:33.904345 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:33.904327 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:20:34.816885 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:34.816851 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" event={"ID":"bb360ea4-948b-4c98-a0c2-edb02c7a8255","Type":"ContainerStarted","Data":"702fee75db9f19db4a421da7a7b72e4a50c4e7b83835d15267165f5abc3788a4"} Jul 09 10:20:34.817058 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:34.816908 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" event={"ID":"bb360ea4-948b-4c98-a0c2-edb02c7a8255","Type":"ContainerStarted","Data":"c86db9ebb27848cd22bcba66159646aef7280f61a5991cd1d4e12e3e450cf913"} Jul 09 10:20:34.841399 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:34.841353 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" podStartSLOduration=1.841337404 podStartE2EDuration="1.841337404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:20:34.839535247 +0000 UTC m=+1573.037627044" watchObservedRunningTime="2026-07-09 10:20:34.841337404 +0000 UTC m=+1573.039429201" Jul 09 10:20:35.747762 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:35.747727 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:35.751881 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:35.751861 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:35.821079 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:35.821053 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:35.822011 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:35.821992 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:36.888129 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:36.888095 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:37.019708 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:37.019677 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:37.143560 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:37.143483 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:37.328884 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:37.328854 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:37.621356 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:37.621321 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:37.718861 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:37.718828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:40.491435 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:40.491407 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:20:41.441851 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:41.441816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:42.170518 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:42.170484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:42.267553 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:42.267518 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:42.668335 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:42.668305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:48.898688 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:48.898649 2576 generic.go:356] "Generic (PLEG): container finished" podID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerID="702fee75db9f19db4a421da7a7b72e4a50c4e7b83835d15267165f5abc3788a4" exitCode=0 Jul 09 10:20:48.899118 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:48.898726 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" event={"ID":"bb360ea4-948b-4c98-a0c2-edb02c7a8255","Type":"ContainerDied","Data":"702fee75db9f19db4a421da7a7b72e4a50c4e7b83835d15267165f5abc3788a4"} Jul 09 10:20:48.899201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:48.899134 2576 scope.go:117] "RemoveContainer" containerID="702fee75db9f19db4a421da7a7b72e4a50c4e7b83835d15267165f5abc3788a4" Jul 09 10:20:48.900202 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:48.900172 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:20:48.900317 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:48.900238 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:20:48.900317 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:48.900272 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:20:49.152770 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:49.152687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:49.251283 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:49.251253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:49.607735 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:49.607658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:49.905712 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:49.905621 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" event={"ID":"bb360ea4-948b-4c98-a0c2-edb02c7a8255","Type":"ContainerStarted","Data":"d20803169b55ad55a425d6555b4b01bd10f461adeac2805844d3b47904e40b94"} Jul 09 10:20:50.025333 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:50.025295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:50.126269 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:50.126231 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:50.747550 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:50.747515 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:50.748946 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:50.748630 2576 patch_prober.go:28] interesting pod/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 10:20:50.748946 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:50.748690 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 10:20:50.821276 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:50.821248 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:51.747703 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:51.747661 2576 patch_prober.go:28] interesting pod/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 10:20:51.748241 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:51.747728 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 10:20:52.747996 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:52.747953 2576 patch_prober.go:28] interesting pod/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 10:20:52.748518 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:52.748026 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 10:20:53.197436 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:53.197352 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:20:53.747593 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:53.747553 2576 patch_prober.go:28] interesting pod/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 10:20:53.747812 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:53.747620 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 10:20:54.751616 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:54.751574 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:54.752598 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:54.752578 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:20:56.543178 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:56.543128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:56.643551 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:56.643491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:56.839370 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:56.839288 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr"] Jul 09 10:20:56.839544 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:56.839517 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" containerID="cri-o://d20803169b55ad55a425d6555b4b01bd10f461adeac2805844d3b47904e40b94" gracePeriod=30 Jul 09 10:20:56.845905 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:56.845860 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:57.237325 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:57.237295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:57.338008 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:57.337976 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:20:57.556124 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:20:57.556049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:01.964131 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:01.964104 2576 generic.go:356] "Generic (PLEG): container finished" podID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerID="d20803169b55ad55a425d6555b4b01bd10f461adeac2805844d3b47904e40b94" exitCode=0 Jul 09 10:21:01.964499 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:01.964171 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" event={"ID":"bb360ea4-948b-4c98-a0c2-edb02c7a8255","Type":"ContainerDied","Data":"d20803169b55ad55a425d6555b4b01bd10f461adeac2805844d3b47904e40b94"} Jul 09 10:21:01.964499 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:01.964220 2576 scope.go:117] "RemoveContainer" containerID="702fee75db9f19db4a421da7a7b72e4a50c4e7b83835d15267165f5abc3788a4" Jul 09 10:21:02.105271 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.105216 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:21:02.171675 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171649 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-data\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.171834 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171693 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s982h\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-kube-api-access-s982h\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.171834 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171717 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-podinfo\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.171834 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171786 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-socket\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.171834 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171811 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-envoy\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.171834 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171834 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-certs\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.172098 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171871 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-credential-socket\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.172098 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171903 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istiod-ca-cert\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.172098 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.171944 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-token\") pod \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\" (UID: \"bb360ea4-948b-4c98-a0c2-edb02c7a8255\") " Jul 09 10:21:02.172280 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172219 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:21:02.172280 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172237 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:21:02.172436 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172416 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.172509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172442 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.172589 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172565 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:21:02.172644 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172571 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:21:02.172691 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.172657 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:21:02.174028 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.174003 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-kube-api-access-s982h" (OuterVolumeSpecName: "kube-api-access-s982h") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "kube-api-access-s982h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:21:02.174289 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.174272 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:21:02.174378 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.174356 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:21:02.174460 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.174444 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb360ea4-948b-4c98-a0c2-edb02c7a8255" (UID: "bb360ea4-948b-4c98-a0c2-edb02c7a8255"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:21:02.273627 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273588 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.273627 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273618 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s982h\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-kube-api-access-s982h\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.273627 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273632 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.273872 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273643 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.273872 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273654 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.273872 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273665 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.273872 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.273677 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb360ea4-948b-4c98-a0c2-edb02c7a8255-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:21:02.970410 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.970378 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" event={"ID":"bb360ea4-948b-4c98-a0c2-edb02c7a8255","Type":"ContainerDied","Data":"c86db9ebb27848cd22bcba66159646aef7280f61a5991cd1d4e12e3e450cf913"} Jul 09 10:21:02.970860 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.970414 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr" Jul 09 10:21:02.970860 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.970440 2576 scope.go:117] "RemoveContainer" containerID="d20803169b55ad55a425d6555b4b01bd10f461adeac2805844d3b47904e40b94" Jul 09 10:21:02.990095 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.990074 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr"] Jul 09 10:21:02.994595 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:02.994569 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1df781d2-openshift-default-756d4dd5c5-xp9nr"] Jul 09 10:21:03.858642 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:03.858609 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:03.959753 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:03.959719 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:04.060298 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:04.060266 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:04.165281 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:04.165193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:04.293597 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:04.293567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:04.329951 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:04.329922 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" path="/var/lib/kubelet/pods/bb360ea4-948b-4c98-a0c2-edb02c7a8255/volumes" Jul 09 10:21:10.937428 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:10.937395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:11.038688 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:11.038660 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:18.062012 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.061936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:18.338407 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.338324 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r"] Jul 09 10:21:18.338961 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.338940 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" Jul 09 10:21:18.338961 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.338962 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" Jul 09 10:21:18.339090 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.339038 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" Jul 09 10:21:18.342268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.342244 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.344968 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.344950 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7dafaaa7-openshift-default-dockercfg-qmrlt\"" Jul 09 10:21:18.355737 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.355716 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r"] Jul 09 10:21:18.381408 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.381379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:18.413688 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41280d26-b619-419b-9a3b-c2154d3e0292-istiod-ca-cert\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413786 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413707 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41280d26-b619-419b-9a3b-c2154d3e0292-istio-podinfo\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413786 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413726 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-socket\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413870 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9wm\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-kube-api-access-dm9wm\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413914 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413878 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-credential-socket\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413914 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413900 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-certs\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413992 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413916 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-istio-token\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413992 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-envoy\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.413992 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.413957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-data\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515307 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515274 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41280d26-b619-419b-9a3b-c2154d3e0292-istiod-ca-cert\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515477 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515333 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41280d26-b619-419b-9a3b-c2154d3e0292-istio-podinfo\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515477 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-socket\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515477 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9wm\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-kube-api-access-dm9wm\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515597 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-credential-socket\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515597 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515516 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-certs\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515597 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515562 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-istio-token\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515746 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-envoy\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515746 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515643 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-data\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515848 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515827 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-socket\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.515921 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515895 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-certs\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.516004 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.515981 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-credential-socket\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.516060 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.516026 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-data\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.516098 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.516077 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41280d26-b619-419b-9a3b-c2154d3e0292-istiod-ca-cert\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.517883 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.517860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-envoy\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.517981 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.517908 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41280d26-b619-419b-9a3b-c2154d3e0292-istio-podinfo\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.524926 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.524905 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-istio-token\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.525406 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.525380 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9wm\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-kube-api-access-dm9wm\") pod \"e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.612470 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.612357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:18.653461 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.653443 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:18.790355 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.790328 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r"] Jul 09 10:21:18.792203 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:21:18.792176 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41280d26_b619_419b_9a3b_c2154d3e0292.slice/crio-5a931c447adaa0e6e3146389812dd4b350780ae7b0c7d8657a13fb78c0a2aa7f WatchSource:0}: Error finding container 5a931c447adaa0e6e3146389812dd4b350780ae7b0c7d8657a13fb78c0a2aa7f: Status 404 returned error can't find the container with id 5a931c447adaa0e6e3146389812dd4b350780ae7b0c7d8657a13fb78c0a2aa7f Jul 09 10:21:18.794377 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.794336 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:21:18.794453 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.794411 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:21:18.794453 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:18.794445 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:21:19.049872 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:19.049835 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" event={"ID":"41280d26-b619-419b-9a3b-c2154d3e0292","Type":"ContainerStarted","Data":"5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d"} Jul 09 10:21:19.050068 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:19.049898 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" event={"ID":"41280d26-b619-419b-9a3b-c2154d3e0292","Type":"ContainerStarted","Data":"5a931c447adaa0e6e3146389812dd4b350780ae7b0c7d8657a13fb78c0a2aa7f"} Jul 09 10:21:19.073803 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:19.073751 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podStartSLOduration=1.073735413 podStartE2EDuration="1.073735413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:21:19.071268797 +0000 UTC m=+1617.269360595" watchObservedRunningTime="2026-07-09 10:21:19.073735413 +0000 UTC m=+1617.271827188" Jul 09 10:21:19.654349 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:19.654317 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:19.658933 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:19.658910 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:20.055070 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:20.055040 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:20.056016 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:20.055992 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:21.097515 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:21.097482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:21.333422 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:21.333390 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:21.434758 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:21.434671 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:21.535859 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:21.535831 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:22.283360 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:22.283325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:22.383339 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:22.383315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:25.680849 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:25.680814 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:26.688474 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:26.688442 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:21:27.624499 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:27.624461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:27.690817 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:27.690786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:35.056485 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.056446 2576 patch_prober.go:28] interesting pod/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 09 10:21:35.056883 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.056507 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 09 10:21:35.128796 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.128762 2576 generic.go:356] "Generic (PLEG): container finished" podID="41280d26-b619-419b-9a3b-c2154d3e0292" containerID="5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d" exitCode=0 Jul 09 10:21:35.128955 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.128821 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" event={"ID":"41280d26-b619-419b-9a3b-c2154d3e0292","Type":"ContainerDied","Data":"5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d"} Jul 09 10:21:35.129246 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.129232 2576 scope.go:117] "RemoveContainer" containerID="5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d" Jul 09 10:21:35.130328 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.130296 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:21:35.130404 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.130386 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:21:35.130441 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.130429 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:21:35.758534 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:35.758496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:36.135080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:36.134999 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" event={"ID":"41280d26-b619-419b-9a3b-c2154d3e0292","Type":"ContainerStarted","Data":"a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766"} Jul 09 10:21:36.165123 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:36.165089 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:36.654762 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:36.654725 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:36.654970 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:36.654847 2576 patch_prober.go:28] interesting pod/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 09 10:21:36.654970 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:36.654925 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 09 10:21:37.654824 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:37.654372 2576 patch_prober.go:28] interesting pod/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 09 10:21:37.654824 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:37.654437 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 09 10:21:38.622274 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:38.622242 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:38.654916 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:38.654544 2576 patch_prober.go:28] interesting pod/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 09 10:21:38.654916 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:38.654612 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 09 10:21:39.654118 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:39.654079 2576 patch_prober.go:28] interesting pod/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 09 10:21:39.654447 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:39.654419 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 09 10:21:40.658466 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:40.658437 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:40.658853 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:40.658815 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:40.659491 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:40.659471 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:21:42.091416 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:42.091381 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:21:44.081970 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:44.081923 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:44.184210 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:44.184136 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:56.087277 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:56.087241 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:21:58.482589 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:58.482541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:21:59.014290 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:21:59.014258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:04.606732 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:04.606701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:09.684105 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:09.684067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:10.187948 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:10.187914 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:22:46.320541 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:46.320412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:47.998481 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:47.998435 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:49.708543 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:49.708506 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:22:54.963680 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:54.963630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:55.066111 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:55.066071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:55.248529 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:55.248498 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r"] Jul 09 10:22:55.248788 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:55.248744 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" containerID="cri-o://a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766" gracePeriod=30 Jul 09 10:22:55.255282 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:55.255259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:55.661193 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:55.661089 2576 patch_prober.go:28] interesting pod/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 10:22:55.661193 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:55.661163 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 10:22:56.676532 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:56.676500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:56.765452 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:56.765415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:56.882199 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:56.882171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:57.710609 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:57.710575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:57.804282 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:57.804246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:57.906897 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:57.906874 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:22:59.297383 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:59.297342 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:22:59.419532 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:22:59.419486 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:00.485978 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.485949 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:23:00.567521 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567431 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41280d26-b619-419b-9a3b-c2154d3e0292-istiod-ca-cert\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.567521 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567485 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-socket\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.567733 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567554 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-istio-token\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.567733 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567600 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41280d26-b619-419b-9a3b-c2154d3e0292-istio-podinfo\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.567733 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567637 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm9wm\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-kube-api-access-dm9wm\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.567888 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567743 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41280d26-b619-419b-9a3b-c2154d3e0292-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:23:00.567888 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567781 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:23:00.567888 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567821 2576 generic.go:356] "Generic (PLEG): container finished" podID="41280d26-b619-419b-9a3b-c2154d3e0292" containerID="a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766" exitCode=0 Jul 09 10:23:00.567888 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567829 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-envoy\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.568094 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567889 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" Jul 09 10:23:00.568094 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" event={"ID":"41280d26-b619-419b-9a3b-c2154d3e0292","Type":"ContainerDied","Data":"a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766"} Jul 09 10:23:00.568094 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567922 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-data\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.568094 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567948 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r" event={"ID":"41280d26-b619-419b-9a3b-c2154d3e0292","Type":"ContainerDied","Data":"5a931c447adaa0e6e3146389812dd4b350780ae7b0c7d8657a13fb78c0a2aa7f"} Jul 09 10:23:00.568094 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567977 2576 scope.go:117] "RemoveContainer" containerID="a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766" Jul 09 10:23:00.568094 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.567977 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-credential-socket\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.568432 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568296 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:23:00.568432 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568313 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-certs\") pod \"41280d26-b619-419b-9a3b-c2154d3e0292\" (UID: \"41280d26-b619-419b-9a3b-c2154d3e0292\") " Jul 09 10:23:00.568703 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568677 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:23:00.568703 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568697 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.568821 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568720 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41280d26-b619-419b-9a3b-c2154d3e0292-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.568821 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568735 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.568930 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.568913 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-data" (OuterVolumeSpecName: "istio-data") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:23:00.570131 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.570090 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41280d26-b619-419b-9a3b-c2154d3e0292-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:23:00.570287 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.570208 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-istio-token" (OuterVolumeSpecName: "istio-token") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:23:00.570287 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.570276 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:23:00.570464 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.570440 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-kube-api-access-dm9wm" (OuterVolumeSpecName: "kube-api-access-dm9wm") pod "41280d26-b619-419b-9a3b-c2154d3e0292" (UID: "41280d26-b619-419b-9a3b-c2154d3e0292"). InnerVolumeSpecName "kube-api-access-dm9wm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:23:00.588693 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.588675 2576 scope.go:117] "RemoveContainer" containerID="5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d" Jul 09 10:23:00.596081 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.596063 2576 scope.go:117] "RemoveContainer" containerID="a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766" Jul 09 10:23:00.596383 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:23:00.596362 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766\": container with ID starting with a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766 not found: ID does not exist" containerID="a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766" Jul 09 10:23:00.596461 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.596390 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766"} err="failed to get container status \"a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766\": rpc error: code = NotFound desc = could not find container \"a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766\": container with ID starting with a458fba48200be0cfcf0914ebc8dab2f1586c01ea5ed25733787ac94f379b766 not found: ID does not exist" Jul 09 10:23:00.596461 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.596407 2576 scope.go:117] "RemoveContainer" containerID="5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d" Jul 09 10:23:00.596644 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:23:00.596624 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d\": container with ID starting with 5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d not found: ID does not exist" containerID="5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d" Jul 09 10:23:00.596702 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.596654 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d"} err="failed to get container status \"5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d\": rpc error: code = NotFound desc = could not find container \"5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d\": container with ID starting with 5e32b0e1a7f8f05122a4deebae4ed2965eb1c3fa235a176b03ef8887865f871d not found: ID does not exist" Jul 09 10:23:00.670091 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.670070 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.670091 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.670089 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41280d26-b619-419b-9a3b-c2154d3e0292-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.670242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.670098 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm9wm\" (UniqueName: \"kubernetes.io/projected/41280d26-b619-419b-9a3b-c2154d3e0292-kube-api-access-dm9wm\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.670242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.670108 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.670242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.670116 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.670242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.670123 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41280d26-b619-419b-9a3b-c2154d3e0292-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:23:00.910194 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.910145 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r"] Jul 09 10:23:00.917743 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:00.917715 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7dafaaa7-openshift-default-f8877d875-9pl7r"] Jul 09 10:23:02.330092 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:02.330061 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" path="/var/lib/kubelet/pods/41280d26-b619-419b-9a3b-c2154d3e0292/volumes" Jul 09 10:23:06.586761 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:06.586725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:06.661970 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:06.661937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:07.329415 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:07.329381 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:11.101672 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:11.101638 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:23:14.066869 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:14.066832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:14.142447 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:14.142417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:14.221741 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:14.221716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:14.335397 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:14.335327 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:24.913350 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:24.913317 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:25.691053 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:25.691018 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:46.341612 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:46.341578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:53.277966 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:53.277926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:54.019576 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019547 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5"] Jul 09 10:23:54.019937 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019925 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" Jul 09 10:23:54.019982 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019939 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" Jul 09 10:23:54.019982 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019952 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" Jul 09 10:23:54.019982 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019957 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" Jul 09 10:23:54.019982 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019970 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" Jul 09 10:23:54.019982 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.019976 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" Jul 09 10:23:54.020135 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.020039 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" Jul 09 10:23:54.020135 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.020048 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb360ea4-948b-4c98-a0c2-edb02c7a8255" containerName="istio-proxy" Jul 09 10:23:54.020135 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.020056 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41280d26-b619-419b-9a3b-c2154d3e0292" containerName="istio-proxy" Jul 09 10:23:54.022193 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.022176 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.025015 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.024993 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-31eb62-openshift-default-dockercfg-jm2wf\"" Jul 09 10:23:54.030976 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.030942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:54.037346 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.037326 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5"] Jul 09 10:23:54.135049 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-certs\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135385 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-podinfo\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135523 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135418 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-data\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135523 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135444 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-socket\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135523 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135493 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-credential-socket\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135680 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istiod-ca-cert\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135680 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcb6v\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-kube-api-access-jcb6v\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135795 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135724 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-token\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.135888 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.135842 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-envoy\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.136588 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.136536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:54.231481 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.231455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:54.236559 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236541 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-podinfo\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236614 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236578 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-data\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236614 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236597 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-socket\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236722 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-credential-socket\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236776 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236743 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istiod-ca-cert\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236824 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcb6v\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-kube-api-access-jcb6v\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236891 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236870 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-token\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.236954 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-envoy\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.237009 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.236973 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-certs\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.237070 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.237028 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-data\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.237070 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.237057 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-credential-socket\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.237207 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.237100 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-socket\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.237388 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.237368 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-certs\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.237526 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.237504 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istiod-ca-cert\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.239175 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.239130 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-podinfo\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.239263 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.239193 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-envoy\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.247400 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.247373 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcb6v\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-kube-api-access-jcb6v\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.247528 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.247511 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-token\") pod \"e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.332436 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.332378 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:54.477026 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.477006 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5"] Jul 09 10:23:54.478109 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:23:54.478089 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa8f300_9c2c_4999_8200_c5bc2a7505c0.slice/crio-2027f36158ecf31a70de107e20b954afbb0769270652246f0aed98f431089403 WatchSource:0}: Error finding container 2027f36158ecf31a70de107e20b954afbb0769270652246f0aed98f431089403: Status 404 returned error can't find the container with id 2027f36158ecf31a70de107e20b954afbb0769270652246f0aed98f431089403 Jul 09 10:23:54.480376 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.480339 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:23:54.480441 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.480423 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:23:54.480480 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.480463 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:23:54.827804 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.827764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" event={"ID":"2fa8f300-9c2c-4999-8200-c5bc2a7505c0","Type":"ContainerStarted","Data":"b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42"} Jul 09 10:23:54.827968 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.827827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" event={"ID":"2fa8f300-9c2c-4999-8200-c5bc2a7505c0","Type":"ContainerStarted","Data":"2027f36158ecf31a70de107e20b954afbb0769270652246f0aed98f431089403"} Jul 09 10:23:54.859882 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:54.859828 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podStartSLOduration=1.8598143139999999 podStartE2EDuration="1.859814314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:23:54.857535843 +0000 UTC m=+1773.055627653" watchObservedRunningTime="2026-07-09 10:23:54.859814314 +0000 UTC m=+1773.057906111" Jul 09 10:23:55.333430 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:55.333395 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:55.338575 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:55.338548 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:55.832399 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:55.832363 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:55.833253 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:55.833224 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:23:56.938300 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:56.938264 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:57.054964 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:57.054931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:57.151562 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:57.151530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:57.459112 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:57.459080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:57.874480 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:57.874447 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:23:58.047953 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:23:58.047925 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:01.500203 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:01.500170 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:01.915314 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:01.915228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:02.269505 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:02.269471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:02.479494 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:02.479458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:05.288490 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:05.288454 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:24:08.096438 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:08.096405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:08.228261 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:08.228232 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:08.328704 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:08.328671 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:08.428722 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:08.428641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:08.823345 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:08.823310 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:08.922007 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:08.921977 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:11.874349 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:11.874307 2576 patch_prober.go:28] interesting pod/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 10:24:11.874829 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:11.874404 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 10:24:12.707805 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:12.707759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:13.921266 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:13.921236 2576 generic.go:356] "Generic (PLEG): container finished" podID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerID="b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42" exitCode=0 Jul 09 10:24:13.921691 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:13.921315 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" event={"ID":"2fa8f300-9c2c-4999-8200-c5bc2a7505c0","Type":"ContainerDied","Data":"b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42"} Jul 09 10:24:13.921763 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:13.921745 2576 scope.go:117] "RemoveContainer" containerID="b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42" Jul 09 10:24:13.922939 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:13.922901 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:24:13.923011 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:13.922995 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:24:13.923058 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:13.923043 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:24:14.927609 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:14.927530 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" event={"ID":"2fa8f300-9c2c-4999-8200-c5bc2a7505c0","Type":"ContainerStarted","Data":"8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9"} Jul 09 10:24:15.332866 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:15.332832 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:24:15.333036 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:15.332981 2576 patch_prober.go:28] interesting pod/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 09 10:24:15.333036 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:15.333025 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 09 10:24:16.333449 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:16.333418 2576 patch_prober.go:28] interesting pod/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 09 10:24:16.333824 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:16.333468 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 09 10:24:17.333302 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:17.333270 2576 patch_prober.go:28] interesting pod/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 09 10:24:17.333509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:17.333318 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 09 10:24:18.332893 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:18.332854 2576 patch_prober.go:28] interesting pod/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 09 10:24:18.333087 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:18.332911 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 09 10:24:18.392451 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:18.392421 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:24:18.767359 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:18.767309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:19.079809 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:19.079710 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:19.333000 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:19.332909 2576 patch_prober.go:28] interesting pod/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 09 10:24:19.333000 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:19.332975 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 09 10:24:20.336300 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:20.336269 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:24:20.336746 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:20.336547 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:24:20.337196 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:20.337179 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:24:25.332441 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:25.332399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:26.704215 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:26.704171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:30.874309 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:30.874275 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:30.971403 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:30.971373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:32.504400 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:32.504371 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:24:46.621619 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:46.621590 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:24:51.492892 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:51.492852 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:24:55.772973 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:24:55.772938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:05.694029 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:05.693990 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:25:06.473105 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:06.473075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:07.507274 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:07.507246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:07.702941 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:07.702904 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:08.170385 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:08.170354 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:08.270468 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:08.270438 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:08.476202 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:08.476147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:13.258377 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:13.258320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:17.794629 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:17.794590 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:25:40.026556 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:40.026519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:50.883202 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:50.883074 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:51.135408 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:51.135329 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5"] Jul 09 10:25:51.135571 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:51.135515 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" containerID="cri-o://8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9" gracePeriod=30 Jul 09 10:25:51.159433 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:51.159388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:51.685394 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:51.685357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:52.093597 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:52.093560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:52.189459 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:52.189421 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:52.291183 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:52.291134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:52.481419 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:52.481391 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:55.288536 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:55.288495 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:25:56.386570 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.386544 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:25:56.440655 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.440570 2576 generic.go:356] "Generic (PLEG): container finished" podID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerID="8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9" exitCode=0 Jul 09 10:25:56.440655 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.440612 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" Jul 09 10:25:56.440655 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.440639 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" event={"ID":"2fa8f300-9c2c-4999-8200-c5bc2a7505c0","Type":"ContainerDied","Data":"8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9"} Jul 09 10:25:56.440891 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.440688 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5" event={"ID":"2fa8f300-9c2c-4999-8200-c5bc2a7505c0","Type":"ContainerDied","Data":"2027f36158ecf31a70de107e20b954afbb0769270652246f0aed98f431089403"} Jul 09 10:25:56.440891 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.440702 2576 scope.go:117] "RemoveContainer" containerID="8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9" Jul 09 10:25:56.449034 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.449017 2576 scope.go:117] "RemoveContainer" containerID="b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42" Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457487 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-data\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457542 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-podinfo\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457617 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-credential-socket\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457641 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-certs\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457682 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcb6v\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-kube-api-access-jcb6v\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457704 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istiod-ca-cert\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457740 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-envoy\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457762 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-socket\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458080 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.457782 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-token\") pod \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\" (UID: \"2fa8f300-9c2c-4999-8200-c5bc2a7505c0\") " Jul 09 10:25:56.458685 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.458174 2576 scope.go:117] "RemoveContainer" containerID="8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9" Jul 09 10:25:56.458685 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.458343 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:56.458685 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.458548 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:56.458685 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:25:56.458657 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9\": container with ID starting with 8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9 not found: ID does not exist" containerID="8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9" Jul 09 10:25:56.458895 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.458692 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9"} err="failed to get container status \"8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9\": rpc error: code = NotFound desc = could not find container \"8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9\": container with ID starting with 8d64737b6fde4cd0953bb7fea097e34256e73253071854d5a95162ec0732a3e9 not found: ID does not exist" Jul 09 10:25:56.458895 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.458716 2576 scope.go:117] "RemoveContainer" containerID="b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42" Jul 09 10:25:56.458895 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.458867 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:56.459218 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:25:56.459192 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42\": container with ID starting with b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42 not found: ID does not exist" containerID="b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42" Jul 09 10:25:56.459324 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.459230 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42"} err="failed to get container status \"b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42\": rpc error: code = NotFound desc = could not find container \"b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42\": container with ID starting with b4b5aa8cebfb93eb44d109b0ab92eda1d2b76520fe98d0ad1100c3475d860d42 not found: ID does not exist" Jul 09 10:25:56.459454 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.459431 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:25:56.459544 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.459488 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:56.460632 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.460600 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-kube-api-access-jcb6v" (OuterVolumeSpecName: "kube-api-access-jcb6v") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "kube-api-access-jcb6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:25:56.460734 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.460665 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:25:56.460734 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.460670 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:56.461025 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.460998 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "2fa8f300-9c2c-4999-8200-c5bc2a7505c0" (UID: "2fa8f300-9c2c-4999-8200-c5bc2a7505c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:25:56.559103 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559080 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559103 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559101 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559110 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559120 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559128 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcb6v\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-kube-api-access-jcb6v\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559136 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559144 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559168 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.559258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.559176 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fa8f300-9c2c-4999-8200-c5bc2a7505c0-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:25:56.766342 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.766314 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5"] Jul 09 10:25:56.771111 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:56.771089 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-31eb62-openshift-default-6c749c5cb7-h9rn5"] Jul 09 10:25:58.330506 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:58.330479 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" path="/var/lib/kubelet/pods/2fa8f300-9c2c-4999-8200-c5bc2a7505c0/volumes" Jul 09 10:25:58.815616 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:58.815581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:58.899920 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:58.899888 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:59.111284 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:59.111202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:25:59.711511 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:25:59.711482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:00.006299 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006270 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd"] Jul 09 10:26:00.006766 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006749 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" Jul 09 10:26:00.006854 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006767 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" Jul 09 10:26:00.006854 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006794 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" Jul 09 10:26:00.006854 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006800 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" Jul 09 10:26:00.006954 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006869 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" Jul 09 10:26:00.006954 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.006881 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa8f300-9c2c-4999-8200-c5bc2a7505c0" containerName="istio-proxy" Jul 09 10:26:00.009836 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.009820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.012635 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.012616 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-73cf2e-openshift-default-dockercfg-56294\"" Jul 09 10:26:00.026407 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.026382 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd"] Jul 09 10:26:00.037689 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.037670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:00.090141 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090113 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-envoy\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090260 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090144 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istiod-ca-cert\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090328 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-podinfo\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090328 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-socket\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090433 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090355 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-certs\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090433 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090422 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvmz\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-kube-api-access-qgvmz\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090527 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090450 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-credential-socket\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090527 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090491 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-token\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.090637 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.090547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-data\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.191809 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.191774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-podinfo\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.191997 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.191834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-socket\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.191997 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.191999 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-certs\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192140 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192063 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvmz\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-kube-api-access-qgvmz\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192140 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-credential-socket\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192140 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-token\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192341 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192191 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-data\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192341 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192253 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-socket\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192341 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-envoy\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192341 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192327 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istiod-ca-cert\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192535 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192349 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-certs\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192535 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-credential-socket\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192678 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192658 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-data\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.192955 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.192939 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istiod-ca-cert\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.194329 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.194299 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-envoy\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.194432 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.194367 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-podinfo\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.201048 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.201028 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-token\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.201268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.201248 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvmz\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-kube-api-access-qgvmz\") pod \"e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.320007 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.319951 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:00.459222 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.459193 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd"] Jul 09 10:26:00.466812 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.466765 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:00.466916 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.466849 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:00.466916 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.466892 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:00.471915 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.471885 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" event={"ID":"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e","Type":"ContainerStarted","Data":"e2fd03b92956690f0a33b67fbbb0f00a42ba038168f826a667c386c56713605a"} Jul 09 10:26:00.623829 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:00.623746 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:01.479734 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:01.479700 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" event={"ID":"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e","Type":"ContainerStarted","Data":"9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d"} Jul 09 10:26:01.505030 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:01.504983 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podStartSLOduration=2.504966886 podStartE2EDuration="2.504966886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:26:01.50264401 +0000 UTC m=+1899.700735808" watchObservedRunningTime="2026-07-09 10:26:01.504966886 +0000 UTC m=+1899.703058682" Jul 09 10:26:02.320461 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:02.320431 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:02.330083 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:02.330060 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:02.485477 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:02.485448 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:02.486233 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:02.486216 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:03.473001 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:03.472962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:03.561306 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:03.561275 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:03.635814 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:03.635773 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:03.737547 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:03.737453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:04.101272 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:04.101191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:04.201908 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:04.201870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:07.985307 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:07.985269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:08.481262 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:08.481213 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:08.798706 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:08.798631 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:09.000392 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:09.000352 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:09.395192 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:09.395143 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:26:11.686986 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:11.686955 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:11.760449 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:11.760417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:11.852176 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:11.841034 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:11.941988 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:11.941865 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:12.376509 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:12.376459 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:12.478417 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:12.478385 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:16.242238 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:16.242206 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:17.487104 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.487075 2576 patch_prober.go:28] interesting pod/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 09 10:26:17.487524 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.487131 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 09 10:26:17.559445 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.559357 2576 generic.go:356] "Generic (PLEG): container finished" podID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerID="9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d" exitCode=0 Jul 09 10:26:17.559589 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.559444 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" event={"ID":"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e","Type":"ContainerDied","Data":"9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d"} Jul 09 10:26:17.559945 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.559929 2576 scope.go:117] "RemoveContainer" containerID="9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d" Jul 09 10:26:17.561015 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.560987 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:17.561115 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.561065 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:17.561196 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:17.561114 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:18.567923 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:18.567892 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" event={"ID":"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e","Type":"ContainerStarted","Data":"903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5"} Jul 09 10:26:19.321017 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:19.320981 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:19.321219 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:19.321183 2576 patch_prober.go:28] interesting pod/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 09 10:26:19.321268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:19.321238 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 09 10:26:20.321123 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:20.321088 2576 patch_prober.go:28] interesting pod/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 09 10:26:20.321550 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:20.321140 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 09 10:26:21.321141 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:21.321105 2576 patch_prober.go:28] interesting pod/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 09 10:26:21.321541 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:21.321185 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 09 10:26:22.320489 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:22.320455 2576 patch_prober.go:28] interesting pod/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 09 10:26:22.320658 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:22.320511 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 09 10:26:23.324516 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:23.324481 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:23.324982 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:23.324831 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:23.325623 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:23.325598 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:24.421644 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:24.421610 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:26:24.539381 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:24.539352 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:24.926284 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:24.926246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:25.094435 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:25.094400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:25.197635 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:25.197558 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:25.502197 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:25.502142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:31.888735 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:31.888701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:31.988188 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:31.988139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:32.135803 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:32.135768 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd"] Jul 09 10:26:32.136013 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:32.135981 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" containerID="cri-o://903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5" gracePeriod=30 Jul 09 10:26:32.152512 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:32.152455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:32.697856 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:32.697823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:33.072719 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:33.072687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:33.988347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:33.988290 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:34.060198 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:34.060139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:35.890931 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:35.890900 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:26:36.601186 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:36.601132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:36.690138 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:36.690106 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:37.383971 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.383951 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:37.454789 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454726 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-envoy\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.454789 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454770 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgvmz\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-kube-api-access-qgvmz\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.454941 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454793 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-data\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.454941 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454818 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-credential-socket\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.454941 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454857 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-podinfo\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.455093 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454951 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-certs\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.455093 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.454980 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-token\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.455093 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455021 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istiod-ca-cert\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.455093 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455068 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-socket\") pod \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\" (UID: \"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e\") " Jul 09 10:26:37.455339 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455132 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:26:37.455339 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455280 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:26:37.455513 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455488 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:26:37.455680 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455588 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.455757 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455694 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.455757 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455703 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:26:37.455757 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.455717 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:26:37.456929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.456901 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-kube-api-access-qgvmz" (OuterVolumeSpecName: "kube-api-access-qgvmz") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "kube-api-access-qgvmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:26:37.457221 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.457196 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:26:37.457458 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.457433 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:26:37.457540 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.457453 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" (UID: "d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:26:37.556736 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556708 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.556736 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556733 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.556736 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556743 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.556952 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556751 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.556952 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556760 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgvmz\" (UniqueName: \"kubernetes.io/projected/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-kube-api-access-qgvmz\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.556952 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556768 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.556952 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.556776 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:26:37.667690 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.667661 2576 generic.go:356] "Generic (PLEG): container finished" podID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerID="903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5" exitCode=0 Jul 09 10:26:37.667857 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.667710 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" Jul 09 10:26:37.667857 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.667747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" event={"ID":"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e","Type":"ContainerDied","Data":"903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5"} Jul 09 10:26:37.667857 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.667799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd" event={"ID":"d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e","Type":"ContainerDied","Data":"e2fd03b92956690f0a33b67fbbb0f00a42ba038168f826a667c386c56713605a"} Jul 09 10:26:37.667857 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.667821 2576 scope.go:117] "RemoveContainer" containerID="903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5" Jul 09 10:26:37.676912 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.676902 2576 scope.go:117] "RemoveContainer" containerID="9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d" Jul 09 10:26:37.684504 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.684486 2576 scope.go:117] "RemoveContainer" containerID="903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5" Jul 09 10:26:37.684734 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:26:37.684716 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5\": container with ID starting with 903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5 not found: ID does not exist" containerID="903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5" Jul 09 10:26:37.684786 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.684740 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5"} err="failed to get container status \"903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5\": rpc error: code = NotFound desc = could not find container \"903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5\": container with ID starting with 903e9569ec4a73b36e218b1e7f90f85487da3bcef29575eb57c313a9b9f55bc5 not found: ID does not exist" Jul 09 10:26:37.684786 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.684755 2576 scope.go:117] "RemoveContainer" containerID="9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d" Jul 09 10:26:37.684966 ip-10-0-138-234 kubenswrapper[2576]: E0709 10:26:37.684947 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d\": container with ID starting with 9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d not found: ID does not exist" containerID="9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d" Jul 09 10:26:37.685021 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.684979 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d"} err="failed to get container status \"9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d\": rpc error: code = NotFound desc = could not find container \"9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d\": container with ID starting with 9017a44f3eb13b0068ae2fa648a4ece616b60ca84dfb111a9ffad61186b46a4d not found: ID does not exist" Jul 09 10:26:37.693163 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.693126 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd"] Jul 09 10:26:37.698855 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:37.698836 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-73cf2e-openshift-default-75b4cbff59-z8fdd"] Jul 09 10:26:38.330195 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:38.330138 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" path="/var/lib/kubelet/pods/d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e/volumes" Jul 09 10:26:39.585498 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:39.585434 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:39.651206 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:39.651174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:39.865724 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:39.865616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:41.024046 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:41.024013 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:41.105114 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:41.105081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:41.175882 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:41.175843 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:43.820438 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:43.820404 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:43.907020 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:43.906972 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:43.987199 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:43.987161 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:44.087770 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:44.087691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:44.646120 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:44.646086 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:44.741972 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:44.741942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:48.351201 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.351176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:48.704825 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.704726 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:48.987039 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.987009 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g"] Jul 09 10:26:48.987494 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.987480 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" Jul 09 10:26:48.987547 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.987496 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" Jul 09 10:26:48.987583 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.987572 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" Jul 09 10:26:48.987583 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.987581 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" Jul 09 10:26:48.990835 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.990819 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:48.993793 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:48.993773 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a0b44e-openshift-default-dockercfg-hd9w6\"" Jul 09 10:26:49.002381 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.002361 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g"] Jul 09 10:26:49.014431 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.014372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:49.164304 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164262 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-data\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164304 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164314 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-certs\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164358 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-socket\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-envoy\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crfhb\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-kube-api-access-crfhb\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-credential-socket\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164462 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-token\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-podinfo\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.164538 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.164517 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istiod-ca-cert\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.265853 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265770 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-token\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.265853 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265810 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-podinfo\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.265853 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265832 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istiod-ca-cert\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266110 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265875 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-data\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266110 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265899 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-certs\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266110 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265921 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-socket\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266110 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265941 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-envoy\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266110 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.265974 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crfhb\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-kube-api-access-crfhb\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266110 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.266005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-credential-socket\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266408 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.266370 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-data\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266408 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.266397 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-credential-socket\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266523 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.266500 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-socket\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266600 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.266578 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-certs\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.266882 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.266861 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istiod-ca-cert\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.268316 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.268286 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-envoy\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.268413 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.268397 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-podinfo\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.275332 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.275307 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-token\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.275534 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.275509 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crfhb\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-kube-api-access-crfhb\") pod \"e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.302347 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.302328 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:49.437066 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.437029 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g"] Jul 09 10:26:49.454258 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.454094 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:49.458857 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:26:49.458836 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9214cd47_8ff8_4bfb_8c89_ba218c49d933.slice/crio-a39e8314be52768d3a1a1fad1645f133e85329ca0c53af0e208507e9c8414d09 WatchSource:0}: Error finding container a39e8314be52768d3a1a1fad1645f133e85329ca0c53af0e208507e9c8414d09: Status 404 returned error can't find the container with id a39e8314be52768d3a1a1fad1645f133e85329ca0c53af0e208507e9c8414d09 Jul 09 10:26:49.461398 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.461366 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:49.461476 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.461431 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:49.461476 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.461459 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:26:49.734739 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.734706 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" event={"ID":"9214cd47-8ff8-4bfb-8c89-ba218c49d933","Type":"ContainerStarted","Data":"9d1b4d784cd81f99373b89a4cdafba35051a42c7b8f6e77de6d2dcd2663ffbb6"} Jul 09 10:26:49.734913 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.734756 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" event={"ID":"9214cd47-8ff8-4bfb-8c89-ba218c49d933","Type":"ContainerStarted","Data":"a39e8314be52768d3a1a1fad1645f133e85329ca0c53af0e208507e9c8414d09"} Jul 09 10:26:49.762328 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:49.762274 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" podStartSLOduration=1.762258207 podStartE2EDuration="1.762258207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:26:49.760467823 +0000 UTC m=+1947.958559619" watchObservedRunningTime="2026-07-09 10:26:49.762258207 +0000 UTC m=+1947.960350003" Jul 09 10:26:50.093296 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:50.093265 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:26:50.303037 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:50.303004 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:50.307280 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:50.307257 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:50.740849 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:50.740811 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:50.741796 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:50.741780 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:26:51.791412 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:51.791382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:51.896162 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:51.896122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:52.011011 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:52.010981 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:52.251709 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:52.251678 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:52.443530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:52.443492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:52.537716 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:52.537640 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:26:56.240580 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:26:56.240541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:27:04.812790 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:04.812761 2576 generic.go:356] "Generic (PLEG): container finished" podID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerID="9d1b4d784cd81f99373b89a4cdafba35051a42c7b8f6e77de6d2dcd2663ffbb6" exitCode=0 Jul 09 10:27:04.813125 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:04.812806 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" event={"ID":"9214cd47-8ff8-4bfb-8c89-ba218c49d933","Type":"ContainerDied","Data":"9d1b4d784cd81f99373b89a4cdafba35051a42c7b8f6e77de6d2dcd2663ffbb6"} Jul 09 10:27:04.813260 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:04.813178 2576 scope.go:117] "RemoveContainer" containerID="9d1b4d784cd81f99373b89a4cdafba35051a42c7b8f6e77de6d2dcd2663ffbb6" Jul 09 10:27:04.814122 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:04.814093 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:27:04.814213 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:04.814187 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:27:04.814263 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:04.814227 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 10:27:05.398591 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:05.398558 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:27:05.741187 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:05.741136 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:27:05.819292 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:05.819257 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" event={"ID":"9214cd47-8ff8-4bfb-8c89-ba218c49d933","Type":"ContainerStarted","Data":"0fd7bc6bda4c0a886cd64b3ebb969843b2e75fef33608b78616677d81b929868"} Jul 09 10:27:06.303109 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:06.303071 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:27:06.304138 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:06.304110 2576 patch_prober.go:28] interesting pod/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 10:27:06.304282 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:06.304181 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 10:27:06.615990 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:06.615852 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:27:06.714963 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:06.714920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:27:07.303447 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:07.303412 2576 patch_prober.go:28] interesting pod/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 10:27:07.303842 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:07.303483 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 10:27:08.303281 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:08.303244 2576 patch_prober.go:28] interesting pod/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 10:27:08.303464 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:08.303298 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 10:27:09.303223 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:09.303188 2576 patch_prober.go:28] interesting pod/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 10:27:09.303409 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:09.303253 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 10:27:10.307518 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:10.307489 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:27:10.307994 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:10.307873 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:27:10.308594 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:10.308574 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:27:13.307643 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:13.307607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:27:13.407941 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:13.407887 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:27:37.779992 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:37.779955 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:27:37.875996 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:27:37.875952 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:29:39.088675 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:29:39.088635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:01.523428 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:01.523386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:01.981699 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:01.981668 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:02.376184 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:02.376077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:02.640965 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:02.640883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:02.781300 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:02.781262 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:06.092789 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:06.092753 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:30:08.368696 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:08.368666 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:08.437853 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:08.437826 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:08.661284 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:08.661208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:09.127350 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:09.127313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:09.943583 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:09.943547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:10.037353 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:10.037320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:10.111203 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:10.111146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:16.303922 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:16.303845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:16.380470 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:16.380435 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:16.642918 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:16.642830 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g"] Jul 09 10:30:16.643079 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:16.643039 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" containerID="cri-o://0fd7bc6bda4c0a886cd64b3ebb969843b2e75fef33608b78616677d81b929868" gracePeriod=30 Jul 09 10:30:16.733363 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:16.733319 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:17.701127 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:17.701094 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zvh96"] Jul 09 10:30:21.779586 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.779558 2576 generic.go:356] "Generic (PLEG): container finished" podID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerID="0fd7bc6bda4c0a886cd64b3ebb969843b2e75fef33608b78616677d81b929868" exitCode=0 Jul 09 10:30:21.779929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.779629 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" event={"ID":"9214cd47-8ff8-4bfb-8c89-ba218c49d933","Type":"ContainerDied","Data":"0fd7bc6bda4c0a886cd64b3ebb969843b2e75fef33608b78616677d81b929868"} Jul 09 10:30:21.779929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.779693 2576 scope.go:117] "RemoveContainer" containerID="9d1b4d784cd81f99373b89a4cdafba35051a42c7b8f6e77de6d2dcd2663ffbb6" Jul 09 10:30:21.890544 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.890524 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:30:21.999772 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999744 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-socket\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:21.999929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999780 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-data\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:21.999929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999811 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-envoy\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:21.999929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999842 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crfhb\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-kube-api-access-crfhb\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:21.999929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999880 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istiod-ca-cert\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:21.999929 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999897 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-certs\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:22.000232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999956 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-credential-socket\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:22.000232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999978 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-token\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:22.000232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:21.999998 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-podinfo\") pod \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\" (UID: \"9214cd47-8ff8-4bfb-8c89-ba218c49d933\") " Jul 09 10:30:22.000232 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.000038 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:30:22.000455 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.000264 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:30:22.000455 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.000291 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.000455 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.000325 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:30:22.000609 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.000463 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:30:22.000761 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.000736 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-data" (OuterVolumeSpecName: "istio-data") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:30:22.002068 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.002045 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:30:22.002198 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.002095 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-kube-api-access-crfhb" (OuterVolumeSpecName: "kube-api-access-crfhb") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "kube-api-access-crfhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:30:22.002198 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.002110 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-token" (OuterVolumeSpecName: "istio-token") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:30:22.002198 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.002176 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9214cd47-8ff8-4bfb-8c89-ba218c49d933" (UID: "9214cd47-8ff8-4bfb-8c89-ba218c49d933"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 10:30:22.101364 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101337 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-credential-socket\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101364 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101362 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-token\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101371 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-podinfo\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101379 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-data\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101387 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istio-envoy\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101395 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crfhb\" (UniqueName: \"kubernetes.io/projected/9214cd47-8ff8-4bfb-8c89-ba218c49d933-kube-api-access-crfhb\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101403 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9214cd47-8ff8-4bfb-8c89-ba218c49d933-istiod-ca-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.101530 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.101412 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9214cd47-8ff8-4bfb-8c89-ba218c49d933-workload-certs\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\"" Jul 09 10:30:22.785245 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.785205 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" event={"ID":"9214cd47-8ff8-4bfb-8c89-ba218c49d933","Type":"ContainerDied","Data":"a39e8314be52768d3a1a1fad1645f133e85329ca0c53af0e208507e9c8414d09"} Jul 09 10:30:22.785245 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.785228 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g" Jul 09 10:30:22.785778 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.785303 2576 scope.go:117] "RemoveContainer" containerID="0fd7bc6bda4c0a886cd64b3ebb969843b2e75fef33608b78616677d81b929868" Jul 09 10:30:22.812332 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.812307 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g"] Jul 09 10:30:22.818565 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:22.818540 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a0b44e-openshift-default-64dbfd548c-dvt2g"] Jul 09 10:30:23.016292 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:23.016266 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-f8nlk_c81b79de-11aa-4b6e-bfc9-bf85663de440/copy-manifests/0.log" Jul 09 10:30:23.025412 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:23.025392 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-f8nlk_c81b79de-11aa-4b6e-bfc9-bf85663de440/manager/0.log" Jul 09 10:30:24.333536 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:24.333501 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" path="/var/lib/kubelet/pods/9214cd47-8ff8-4bfb-8c89-ba218c49d933/volumes" Jul 09 10:30:25.487777 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:25.487750 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zvh96_b8717516-5e1a-4116-b297-7125c2893642/limitador/0.log" Jul 09 10:30:25.986126 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:25.986097 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678vnp2r_c262fcfc-ee30-4f3a-8de5-e23a6c96f86b/istio-proxy/0.log" Jul 09 10:30:26.115278 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:26.115246 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmbnl_4257f47c-2113-4728-97c3-2089b6266d59/discovery/0.log" Jul 09 10:30:26.493892 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:26.493870 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/1.log" Jul 09 10:30:27.748224 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:27.748196 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-25rck_6db2af6f-f172-4807-b20f-999eb4f96509/main/0.log" Jul 09 10:30:32.035104 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:32.035062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:32.130592 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:32.130556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:36.052874 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:36.052844 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hpl6x_049c3668-cc57-448f-85bf-5f99de584f31/global-pull-secret-syncer/0.log" Jul 09 10:30:36.152178 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:36.152137 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" Jul 09 10:30:36.283389 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:36.283362 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-234.ec2.internal_be7f2ab963a045a1608548ebc092ac92/haproxy/0.log" Jul 09 10:30:42.505514 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:42.505487 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zvh96_b8717516-5e1a-4116-b297-7125c2893642/limitador/0.log" Jul 09 10:30:44.332288 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.332262 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/alertmanager/0.log" Jul 09 10:30:44.363461 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.363435 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/config-reloader/0.log" Jul 09 10:30:44.416297 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.416274 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/kube-rbac-proxy-web/0.log" Jul 09 10:30:44.449521 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.449497 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/kube-rbac-proxy/0.log" Jul 09 10:30:44.495392 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.495365 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/kube-rbac-proxy-metric/0.log" Jul 09 10:30:44.530855 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.530831 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/prom-label-proxy/0.log" Jul 09 10:30:44.567747 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.567679 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_10051a97-9ad4-44e5-9309-eb725e42083c/init-config-reloader/0.log" Jul 09 10:30:44.707488 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.707457 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8v4wr_f2bd1c6f-60e5-435f-952f-0e15f618b343/kube-state-metrics/0.log" Jul 09 10:30:44.758283 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.758252 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8v4wr_f2bd1c6f-60e5-435f-952f-0e15f618b343/kube-rbac-proxy-main/0.log" Jul 09 10:30:44.814535 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.814508 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8v4wr_f2bd1c6f-60e5-435f-952f-0e15f618b343/kube-rbac-proxy-self/0.log" Jul 09 10:30:44.865814 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.865751 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7bc595d68f-jvnrv_a6d6fbd1-3bac-402a-91b9-45ddeaca2257/metrics-server/0.log" Jul 09 10:30:44.918015 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.917989 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-78dh4_0e025dbb-4bca-492f-a806-03b1575516bb/monitoring-plugin/0.log" Jul 09 10:30:44.952732 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.952705 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-brsxs_f927ac61-9c40-49b4-860a-7b44a3f16226/node-exporter/0.log" Jul 09 10:30:44.982136 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:44.982105 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-brsxs_f927ac61-9c40-49b4-860a-7b44a3f16226/kube-rbac-proxy/0.log" Jul 09 10:30:45.012654 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.012625 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-brsxs_f927ac61-9c40-49b4-860a-7b44a3f16226/init-textfile/0.log" Jul 09 10:30:45.264139 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.264115 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-q9tv2_e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d/kube-rbac-proxy-main/0.log" Jul 09 10:30:45.293997 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.293972 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-q9tv2_e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d/kube-rbac-proxy-self/0.log" Jul 09 10:30:45.324705 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.324678 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-q9tv2_e6e53bdc-ec52-4e69-8e34-1f40b53c6e9d/openshift-state-metrics/0.log" Jul 09 10:30:45.367689 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.367670 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/prometheus/0.log" Jul 09 10:30:45.409597 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.409578 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/config-reloader/0.log" Jul 09 10:30:45.442311 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.442292 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/thanos-sidecar/0.log" Jul 09 10:30:45.493391 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.493373 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/kube-rbac-proxy-web/0.log" Jul 09 10:30:45.525633 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.525582 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/kube-rbac-proxy/0.log" Jul 09 10:30:45.555184 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.555142 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/kube-rbac-proxy-thanos/0.log" Jul 09 10:30:45.583145 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.583127 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0051aa0e-be16-489e-9939-a10fd1e26f96/init-config-reloader/0.log" Jul 09 10:30:45.621661 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.621637 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-n6trj_7c3c684d-c416-467e-8229-c072540001f1/prometheus-operator/0.log" Jul 09 10:30:45.649030 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.649008 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-n6trj_7c3c684d-c416-467e-8229-c072540001f1/kube-rbac-proxy/0.log" Jul 09 10:30:45.682078 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.682053 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-d97xr_614004be-6dda-4486-8120-cfc0bfea0536/prometheus-operator-admission-webhook/0.log" Jul 09 10:30:45.723295 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.723270 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7466fd7d65-fzfwp_6af3cdce-e40c-4843-af80-cff1d2939ef1/telemeter-client/0.log" Jul 09 10:30:45.754905 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.754875 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7466fd7d65-fzfwp_6af3cdce-e40c-4843-af80-cff1d2939ef1/reload/0.log" Jul 09 10:30:45.783599 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.783537 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7466fd7d65-fzfwp_6af3cdce-e40c-4843-af80-cff1d2939ef1/kube-rbac-proxy/0.log" Jul 09 10:30:45.834507 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.834487 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d8cddbb6f-4rm6j_dc00daab-1825-4791-9274-ad4727a6640c/thanos-query/0.log" Jul 09 10:30:45.866452 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.866424 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d8cddbb6f-4rm6j_dc00daab-1825-4791-9274-ad4727a6640c/kube-rbac-proxy-web/0.log" Jul 09 10:30:45.900633 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.900611 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d8cddbb6f-4rm6j_dc00daab-1825-4791-9274-ad4727a6640c/kube-rbac-proxy/0.log" Jul 09 10:30:45.932880 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.932859 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d8cddbb6f-4rm6j_dc00daab-1825-4791-9274-ad4727a6640c/prom-label-proxy/0.log" Jul 09 10:30:45.964440 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.964411 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d8cddbb6f-4rm6j_dc00daab-1825-4791-9274-ad4727a6640c/kube-rbac-proxy-rules/0.log" Jul 09 10:30:45.993388 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:45.993369 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d8cddbb6f-4rm6j_dc00daab-1825-4791-9274-ad4727a6640c/kube-rbac-proxy-metrics/0.log" Jul 09 10:30:46.312645 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.312611 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt"] Jul 09 10:30:46.313046 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313033 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" Jul 09 10:30:46.313134 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313049 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b6112e-2b7e-4a66-a555-4b5e78f2bb0e" containerName="istio-proxy" Jul 09 10:30:46.313134 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313063 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" Jul 09 10:30:46.313134 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313069 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" Jul 09 10:30:46.313134 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313077 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" Jul 09 10:30:46.313134 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313082 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" Jul 09 10:30:46.313318 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.313163 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9214cd47-8ff8-4bfb-8c89-ba218c49d933" containerName="istio-proxy" Jul 09 10:30:46.316740 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.316723 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.319669 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.319648 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qjnrh\"/\"openshift-service-ca.crt\"" Jul 09 10:30:46.319880 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.319866 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qjnrh\"/\"default-dockercfg-q7c7q\"" Jul 09 10:30:46.320863 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.320846 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qjnrh\"/\"kube-root-ca.crt\"" Jul 09 10:30:46.331383 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.331365 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt"] Jul 09 10:30:46.417453 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.417427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrn6s\" (UniqueName: \"kubernetes.io/projected/d356de44-5858-4672-ba8d-0e10d9486f33-kube-api-access-mrn6s\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.417835 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.417477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-proc\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.417835 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.417546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-sys\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.417835 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.417568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-podres\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.417835 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.417690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-lib-modules\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518250 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-proc\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518369 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518264 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-sys\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518369 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518288 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-podres\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518369 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518355 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-sys\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518369 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-lib-modules\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518562 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518355 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-proc\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518562 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrn6s\" (UniqueName: \"kubernetes.io/projected/d356de44-5858-4672-ba8d-0e10d9486f33-kube-api-access-mrn6s\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518562 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-podres\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.518562 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.518486 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d356de44-5858-4672-ba8d-0e10d9486f33-lib-modules\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.528108 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.528087 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrn6s\" (UniqueName: \"kubernetes.io/projected/d356de44-5858-4672-ba8d-0e10d9486f33-kube-api-access-mrn6s\") pod \"perf-node-gather-daemonset-qcmdt\" (UID: \"d356de44-5858-4672-ba8d-0e10d9486f33\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.626207 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.626111 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.756267 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.756244 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt"] Jul 09 10:30:46.758262 ip-10-0-138-234 kubenswrapper[2576]: W0709 10:30:46.758226 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd356de44_5858_4672_ba8d_0e10d9486f33.slice/crio-5d60bb72b76ab3fd4539244d5c691ecbe0ff0c0ce38b12631354d85a7c57c0df WatchSource:0}: Error finding container 5d60bb72b76ab3fd4539244d5c691ecbe0ff0c0ce38b12631354d85a7c57c0df: Status 404 returned error can't find the container with id 5d60bb72b76ab3fd4539244d5c691ecbe0ff0c0ce38b12631354d85a7c57c0df Jul 09 10:30:46.901268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.901185 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" event={"ID":"d356de44-5858-4672-ba8d-0e10d9486f33","Type":"ContainerStarted","Data":"4cc4da00efc96a42294549b3f9a57a3dd2156251e32c6d57078e766aef4fe7f8"} Jul 09 10:30:46.901268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.901240 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" event={"ID":"d356de44-5858-4672-ba8d-0e10d9486f33","Type":"ContainerStarted","Data":"5d60bb72b76ab3fd4539244d5c691ecbe0ff0c0ce38b12631354d85a7c57c0df"} Jul 09 10:30:46.901268 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.901266 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:46.927457 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:46.927410 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" podStartSLOduration=0.927396285 podStartE2EDuration="927.396285ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 10:30:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:30:46.92525411 +0000 UTC m=+2185.123345918" watchObservedRunningTime="2026-07-09 10:30:46.927396285 +0000 UTC m=+2185.125488073" Jul 09 10:30:48.325441 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:48.325404 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-hxqvh_4e7b2884-a6b1-48ab-a959-e360e4fa153a/download-server/0.log" Jul 09 10:30:50.191470 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:50.191437 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x6k8c_cff2144a-a21f-4900-9081-7078900fb02f/dns/0.log" Jul 09 10:30:50.226884 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:50.226862 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x6k8c_cff2144a-a21f-4900-9081-7078900fb02f/kube-rbac-proxy/0.log" Jul 09 10:30:50.335242 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:50.335216 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wvb64_0115514f-42d4-40a8-94ea-cfb3069fa696/dns-node-resolver/0.log" Jul 09 10:30:51.047500 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:51.047463 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvxlb_3014cfa9-cf7d-4b35-bd09-ff468ba689a2/node-ca/0.log" Jul 09 10:30:52.180901 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:52.180870 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678vnp2r_c262fcfc-ee30-4f3a-8de5-e23a6c96f86b/istio-proxy/0.log" Jul 09 10:30:52.284833 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:52.284810 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmbnl_4257f47c-2113-4728-97c3-2089b6266d59/discovery/0.log" Jul 09 10:30:52.387706 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:52.387680 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/0.log" Jul 09 10:30:52.414226 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:52.414205 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/1.log" Jul 09 10:30:52.915375 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:52.915352 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-qcmdt" Jul 09 10:30:53.384913 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:53.384889 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gcvsp_d6336c87-140b-4171-a441-2a99d62fbb1e/serve-healthcheck-canary/0.log" Jul 09 10:30:54.009077 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:54.009048 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d2ptr_eead6a6b-baf5-44ec-a31c-a7aacb7a7423/kube-rbac-proxy/0.log" Jul 09 10:30:54.048501 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:54.048480 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d2ptr_eead6a6b-baf5-44ec-a31c-a7aacb7a7423/exporter/0.log" Jul 09 10:30:54.081965 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:54.081952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d2ptr_eead6a6b-baf5-44ec-a31c-a7aacb7a7423/extractor/0.log" Jul 09 10:30:56.358248 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:56.358221 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678vnp2r_c262fcfc-ee30-4f3a-8de5-e23a6c96f86b/istio-proxy/0.log" Jul 09 10:30:56.476262 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:56.476201 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmbnl_4257f47c-2113-4728-97c3-2089b6266d59/discovery/0.log" Jul 09 10:30:56.541446 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:56.541424 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/0.log" Jul 09 10:30:56.565904 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:56.565883 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/1.log" Jul 09 10:30:57.301006 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:57.300977 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-f8nlk_c81b79de-11aa-4b6e-bfc9-bf85663de440/manager/0.log" Jul 09 10:30:57.351855 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:57.351830 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-f8nlk_c81b79de-11aa-4b6e-bfc9-bf85663de440/copy-manifests/0.log" Jul 09 10:30:58.105144 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:58.105029 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678vnp2r_c262fcfc-ee30-4f3a-8de5-e23a6c96f86b/istio-proxy/0.log" Jul 09 10:30:58.205840 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:58.205820 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmbnl_4257f47c-2113-4728-97c3-2089b6266d59/discovery/0.log" Jul 09 10:30:58.299845 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:58.299819 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/0.log" Jul 09 10:30:58.326230 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:58.326205 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-x5x5h_0454bc0d-d0dd-4dc1-833d-74ebf75a78ff/istio-proxy/1.log" Jul 09 10:30:58.994506 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:58.994478 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c844974c9-r7rvm_2514eb6f-afde-4520-a3fd-e3f16dee83d0/manager/0.log" Jul 09 10:30:59.031601 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:59.031571 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-px2wr_a66bc06a-438c-462a-bfda-00df45bb37ff/openshift-lws-operator/0.log" Jul 09 10:30:59.252677 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:59.252604 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024 Jul 09 10:30:59.324279 ip-10-0-138-234 kubenswrapper[2576]: I0709 10:30:59.324252 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8prlt_14190e32-666c-4490-b533-adafc8c34b41/konnectivity-agent/0.log" limit=1024