Sep 10 17:39:06.119517 ip-10-0-134-57 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 17:39:06.119528 ip-10-0-134-57 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 17:39:06.119537 ip-10-0-134-57 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 17:39:06.119739 ip-10-0-134-57 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 17:39:17.480222 ip-10-0-134-57 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 17:39:17.480238 ip-10-0-134-57 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 6dbba7fa2a284af9a575838f1eb68dc1 -- Sep 10 17:41:11.004516 ip-10-0-134-57 systemd[1]: Starting Kubernetes Kubelet... Sep 10 17:41:11.392798 ip-10-0-134-57 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 17:41:11.392798 ip-10-0-134-57 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 17:41:11.392798 ip-10-0-134-57 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 17:41:11.392798 ip-10-0-134-57 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 17:41:11.392798 ip-10-0-134-57 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 17:41:11.393499 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.392858 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 17:41:11.396336 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396320 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:41:11.396336 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396335 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396339 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396343 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396346 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396350 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396354 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396357 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396361 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396364 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396367 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396370 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396373 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396376 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396378 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396381 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396384 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396387 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396389 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396392 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396395 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:41:11.396401 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396398 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396402 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396406 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396410 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396413 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396416 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396425 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396428 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396431 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396433 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396436 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396439 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396441 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396444 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396447 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396450 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396453 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396455 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396458 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:41:11.396953 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396461 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396464 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396466 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396470 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396473 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396476 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396480 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396483 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396486 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396489 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396492 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396494 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396497 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396501 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396504 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396508 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396511 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396514 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396517 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:41:11.397458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396519 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396522 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396525 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396528 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396533 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396536 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396539 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396541 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396544 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396547 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396550 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396552 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396555 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396557 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396560 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396563 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396565 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396568 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396570 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396573 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:41:11.397930 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396576 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396579 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396582 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396584 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396587 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396589 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396592 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396595 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396598 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396601 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396604 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396607 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396609 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396612 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396615 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396617 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396620 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396622 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396625 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.396627 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:41:11.398430 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397363 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397371 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397375 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397378 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397381 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397384 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397386 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397389 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397392 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397395 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397398 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397402 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397405 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397409 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397412 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397414 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397417 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397420 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397423 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397425 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:41:11.398913 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397429 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397432 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397434 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397437 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397440 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397443 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397445 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397450 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397453 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397456 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397459 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397462 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397465 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397468 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397471 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397473 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397476 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397479 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397483 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:41:11.399431 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397486 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397489 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397492 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397495 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397498 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397501 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397504 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397507 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397510 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397512 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397515 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397519 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397522 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397525 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397528 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397531 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397534 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397537 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397539 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397542 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:41:11.399895 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397545 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397548 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397550 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397553 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397556 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397559 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397561 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397564 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397567 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397569 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397572 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397575 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397578 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397580 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397583 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397585 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397588 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397591 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397594 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397597 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:41:11.400404 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397605 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397608 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397611 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397614 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397617 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397620 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397622 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397625 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397628 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397630 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397633 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397636 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397638 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397641 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397643 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397646 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397648 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397651 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397653 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.397656 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:41:11.400886 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398049 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398054 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398057 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398060 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398063 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398066 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398069 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398071 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398074 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398077 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398079 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398082 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398085 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398088 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398091 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398096 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398099 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398102 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398105 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398108 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:41:11.401400 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398112 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398116 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398119 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398121 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398124 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398127 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398130 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398133 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398135 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398138 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398141 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398143 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398147 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398151 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398153 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398156 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398159 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398162 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398165 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:41:11.401887 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398167 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398170 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398174 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398177 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398180 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398183 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398204 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398208 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398215 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398218 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398232 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398235 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398238 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398241 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398245 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398248 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398251 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398254 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398257 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:41:11.402359 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398259 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398262 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398265 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398268 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398271 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398273 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398276 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398278 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398281 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398284 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398286 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398289 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398291 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398295 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398297 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398300 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398303 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398306 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398309 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398312 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:41:11.402854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398320 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398325 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398327 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398330 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398332 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398335 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398338 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398340 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398343 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398345 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398348 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398351 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398353 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398356 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398358 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398361 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398363 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398366 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398369 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398371 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:41:11.403356 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.398374 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398437 2565 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398470 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398479 2565 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398484 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398488 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398492 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398498 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398502 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398507 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398511 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398517 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398522 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398536 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398540 2565 flags.go:64] FLAG: --cgroup-root="" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398543 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398546 2565 flags.go:64] FLAG: --client-ca-file="" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398549 2565 flags.go:64] FLAG: --cloud-provider="external" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398552 2565 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398558 2565 flags.go:64] FLAG: --cluster-domain="" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398561 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398564 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398567 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398571 2565 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 17:41:11.403839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398575 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398578 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398580 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398584 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398587 2565 flags.go:64] FLAG: --contention-profiling="false" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398590 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398593 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398596 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398599 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398603 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398606 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398609 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398612 2565 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398615 2565 flags.go:64] FLAG: --enable-server="true" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398618 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398626 2565 flags.go:64] FLAG: --event-burst="100" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398629 2565 flags.go:64] FLAG: --event-qps="50" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398632 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398635 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398638 2565 flags.go:64] FLAG: --eviction-hard="" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398646 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398649 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398654 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398657 2565 flags.go:64] FLAG: --eviction-soft="" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398660 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 17:41:11.404466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398663 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398666 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398668 2565 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398672 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398674 2565 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398677 2565 flags.go:64] FLAG: --feature-gates="" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398681 2565 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398684 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398687 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398690 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398693 2565 flags.go:64] FLAG: --healthz-port="10248" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398697 2565 flags.go:64] FLAG: --help="false" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398699 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398702 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398706 2565 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398709 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398712 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398715 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398718 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398721 2565 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398724 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398727 2565 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398731 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398734 2565 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 17:41:11.405097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398737 2565 flags.go:64] FLAG: --kube-reserved="" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398740 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398742 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398745 2565 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398748 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398751 2565 flags.go:64] FLAG: --lock-file="" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398754 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398758 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398761 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398766 2565 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398769 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398772 2565 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398775 2565 flags.go:64] FLAG: --logging-format="text" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398778 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398781 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398784 2565 flags.go:64] FLAG: --manifest-url="" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398787 2565 flags.go:64] FLAG: --manifest-url-header="" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398791 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398794 2565 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398799 2565 flags.go:64] FLAG: --max-pods="110" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398802 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398805 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398808 2565 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398811 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398814 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 17:41:11.405763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398817 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398820 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398827 2565 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398830 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398833 2565 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398837 2565 flags.go:64] FLAG: --pod-cidr="" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398840 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398845 2565 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398848 2565 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398852 2565 flags.go:64] FLAG: --pods-per-core="0" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398855 2565 flags.go:64] FLAG: --port="10250" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398858 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398861 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03ec18df78d44d7f9" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398864 2565 flags.go:64] FLAG: --qos-reserved="" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398867 2565 flags.go:64] FLAG: --read-only-port="10255" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398870 2565 flags.go:64] FLAG: --register-node="true" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398873 2565 flags.go:64] FLAG: --register-with-taints="" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398877 2565 flags.go:64] FLAG: --registry-burst="10" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398880 2565 flags.go:64] FLAG: --registry-qps="5" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398883 2565 flags.go:64] FLAG: --reserved-cpus="" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398885 2565 flags.go:64] FLAG: --reserved-memory="" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398889 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398892 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398895 2565 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398898 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 17:41:11.406392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398901 2565 flags.go:64] FLAG: --runonce="false" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398904 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398907 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398912 2565 flags.go:64] FLAG: --seccomp-default="false" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398915 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398919 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398923 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398926 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398929 2565 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398932 2565 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398935 2565 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398938 2565 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398941 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398945 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398948 2565 flags.go:64] FLAG: --system-cgroups="" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398951 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398957 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398960 2565 flags.go:64] FLAG: --tls-cert-file="" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398963 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398969 2565 flags.go:64] FLAG: --tls-min-version="" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398972 2565 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398975 2565 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398979 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398982 2565 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.398985 2565 flags.go:64] FLAG: --v="2" Sep 10 17:41:11.407059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.399002 2565 flags.go:64] FLAG: --version="false" Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.399007 2565 flags.go:64] FLAG: --vmodule="" Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.399011 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.399015 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399141 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399145 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399148 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399151 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399154 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399157 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399161 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399165 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399170 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399173 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399176 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399179 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399182 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399185 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399188 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399190 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:41:11.407691 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399195 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399198 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399201 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399204 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399207 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399209 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399212 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399215 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399218 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399247 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399252 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399255 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399265 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399268 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399271 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399274 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399277 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399280 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399283 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399285 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:41:11.408237 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399288 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399292 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399297 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399301 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399304 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399306 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399309 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399312 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399316 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399318 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399321 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399324 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399328 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399331 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399333 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399336 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399339 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399341 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399344 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:41:11.408734 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399347 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399350 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399352 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399355 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399358 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399361 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399363 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399366 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399368 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399371 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399374 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399376 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399379 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399381 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399384 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399388 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399390 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399393 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399396 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399398 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:41:11.409200 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399401 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399404 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399406 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399409 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399411 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399415 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399418 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399421 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399424 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399426 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399429 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399432 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399435 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399437 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399440 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399443 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399446 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399449 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399451 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399455 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:41:11.409780 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399459 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399462 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399465 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.399468 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.400006 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.408467 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.408484 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.410241 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 17:41:11.410288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.410252 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410318 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410324 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410328 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410331 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410334 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410336 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410339 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410342 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410345 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410348 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410350 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410354 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410356 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410359 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410362 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410365 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410368 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410370 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410373 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:41:11.410488 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410376 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410378 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410381 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410384 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410386 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410389 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410392 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410394 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410397 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410400 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410408 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410411 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410413 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410416 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410419 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410422 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410426 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410428 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410431 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410434 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:41:11.410980 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410437 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410440 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410442 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410445 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410448 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410451 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410453 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410456 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410459 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410462 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410464 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410467 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410470 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410472 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410475 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410478 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410481 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410485 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410488 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410491 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:41:11.411522 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410494 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410496 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410499 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410509 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410513 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410516 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410520 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410523 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410526 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410529 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410531 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410534 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410537 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410540 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410542 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410545 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410548 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410550 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410553 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:41:11.412023 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410555 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410558 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410561 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410563 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410567 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410570 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410573 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410576 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410579 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410582 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410585 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410587 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410591 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410593 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410596 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410598 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410601 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410609 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410612 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:41:11.412505 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410615 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410618 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.410624 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410763 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410768 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410772 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410775 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410778 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410781 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410784 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410787 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410790 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410793 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410796 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410799 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:41:11.412990 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410802 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410804 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410807 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410810 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410812 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410815 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410817 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410820 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410823 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410825 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410828 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410831 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410833 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410836 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410838 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410841 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410851 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410854 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410857 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:41:11.413474 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410860 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410869 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410872 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410874 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410877 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410880 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410884 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410888 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410891 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410894 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410897 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410900 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410902 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410905 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410907 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410910 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410913 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410915 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410918 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410920 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:41:11.413927 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410923 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410926 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410930 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410933 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410936 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410939 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410942 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410945 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410947 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410950 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410958 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410961 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410964 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410967 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410970 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410972 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410975 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410977 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410980 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:41:11.414479 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410983 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410985 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410988 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410991 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410994 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410997 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.410999 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411002 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411004 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411007 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411009 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411012 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411014 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411017 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411019 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411023 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411027 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411030 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411032 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411035 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:41:11.415086 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411038 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411040 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411043 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411045 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411054 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411057 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411060 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411062 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:11.411065 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.411070 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.411196 2565 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.413910 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.414749 2565 server.go:1019] "Starting client certificate rotation" Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.414865 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 17:41:11.415594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.414919 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 17:41:11.418956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.418924 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 17:41:11.420468 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.420444 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 17:41:11.431633 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.431612 2565 log.go:25] "Validated CRI v1 runtime API" Sep 10 17:41:11.434577 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.434563 2565 log.go:25] "Validated CRI v1 image API" Sep 10 17:41:11.435429 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.435404 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 17:41:11.441131 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.441102 2565 fs.go:135] Filesystem UUIDs: map[13c80ec5-47bc-42a5-9cb4-232c63da941c:/dev/nvme0n1p3 39d3188a-9388-42b3-bed9-bc06c8208f97:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 10 17:41:11.441207 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.441129 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 17:41:11.446439 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.446306 2565 manager.go:216] Machine: {Timestamp:2026-09-10 17:41:11.445026801 +0000 UTC m=+0.326751933 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100198 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2822083ebc1c154754460a1853074b SystemUUID:ec282208-3ebc-1c15-4754-460a1853074b BootID:6dbba7fa-2a28-4af9-a575-838f1eb68dc1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3d:81:de:40:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3d:81:de:40:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:ce:f9:18:70:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 17:41:11.446439 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.446427 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 17:41:11.446593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.446512 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 17:41:11.446940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.446922 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 17:41:11.448814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.448788 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 17:41:11.448988 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.448821 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-57.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 17:41:11.450256 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.450243 2565 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 17:41:11.450311 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.450260 2565 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 17:41:11.450311 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.450272 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 17:41:11.450861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.450850 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 17:41:11.451586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.451574 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 17:41:11.454895 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.454880 2565 state_mem.go:36] "Initialized new in-memory state store" Sep 10 17:41:11.455033 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455022 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 17:41:11.455094 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455061 2565 kubelet.go:486] "Attempting to sync node with API server" Sep 10 17:41:11.455094 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455075 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 17:41:11.455094 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455094 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 17:41:11.455239 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455107 2565 kubelet.go:398] "Adding apiserver pod source" Sep 10 17:41:11.455239 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455131 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 17:41:11.455613 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455599 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 17:41:11.455681 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.455622 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 17:41:11.459392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.459373 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 17:41:11.461591 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.461573 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 17:41:11.461689 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.461672 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 17:41:11.463836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463816 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 17:41:11.463919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463845 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 17:41:11.463919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463882 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 17:41:11.463919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463913 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463924 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463939 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463954 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463960 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463976 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.463988 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.464001 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 17:41:11.464006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.464009 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 17:41:11.464893 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.464879 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 17:41:11.464893 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.464895 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 17:41:11.469024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.469010 2565 server.go:1268] "Started kubelet" Sep 10 17:41:11.469120 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.469030 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-57.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 17:41:11.469120 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.469045 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 17:41:11.469120 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.469096 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 17:41:11.469120 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.469111 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 17:41:11.469322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.469199 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 17:41:11.469322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.469292 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 17:41:11.470478 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.470462 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 17:41:11.471118 ip-10-0-134-57 systemd[1]: Started Kubernetes Kubelet. Sep 10 17:41:11.471889 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.471843 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwxtj" Sep 10 17:41:11.472294 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.472276 2565 server.go:310] "Adding debug handlers to kubelet server" Sep 10 17:41:11.473130 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.473099 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 17:41:11.474819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.474804 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 17:41:11.474881 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.474819 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 17:41:11.474946 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.474924 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:11.475463 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.475414 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 17:41:11.475673 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.475657 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 17:41:11.475673 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.475674 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 17:41:11.475788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.475774 2565 reconstruct.go:97] "Volume reconstruction finished" Sep 10 17:41:11.475930 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.475789 2565 reconciler.go:29] "Reconciler: start to sync state" Sep 10 17:41:11.476589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.476566 2565 factory.go:55] Registering systemd factory Sep 10 17:41:11.476589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.476586 2565 factory.go:223] Registration of the systemd container factory successfully Sep 10 17:41:11.477946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.477714 2565 factory.go:153] Registering CRI-O factory Sep 10 17:41:11.477946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.477731 2565 factory.go:223] Registration of the crio container factory successfully Sep 10 17:41:11.477946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.477786 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 17:41:11.477946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.477805 2565 factory.go:103] Registering Raw factory Sep 10 17:41:11.477946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.477819 2565 manager.go:1196] Started watching for new ooms in manager Sep 10 17:41:11.478377 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.476717 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-57.ec2.internal.18d40679a3088c3b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-57.ec2.internal,UID:ip-10-0-134-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-57.ec2.internal,},FirstTimestamp:2026-09-10 17:41:11.468985403 +0000 UTC m=+0.350710535,LastTimestamp:2026-09-10 17:41:11.468985403 +0000 UTC m=+0.350710535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-57.ec2.internal,}" Sep 10 17:41:11.478799 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.478783 2565 manager.go:349] Starting recovery of all containers Sep 10 17:41:11.480057 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.480038 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwxtj" Sep 10 17:41:11.483860 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.483825 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 17:41:11.485542 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.485512 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 17:41:11.485999 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.485813 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 17:41:11.488448 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.488429 2565 manager.go:354] Recovery completed Sep 10 17:41:11.496072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.496036 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.498026 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498007 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.498219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498044 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.498219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498060 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.498769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498754 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 17:41:11.498769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498768 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 17:41:11.498867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498787 2565 state_mem.go:36] "Initialized new in-memory state store" Sep 10 17:41:11.498867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.498798 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.499653 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.499637 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.499743 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.499669 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.499743 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.499681 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.499743 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.499728 2565 policy_none.go:49] "None policy: Start" Sep 10 17:41:11.499874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.499746 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 17:41:11.499874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.499761 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 17:41:11.500668 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.500655 2565 policy_none.go:47] "Start" Sep 10 17:41:11.527198 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527180 2565 manager.go:337] "Starting Device Plugin manager" Sep 10 17:41:11.527324 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.527217 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 17:41:11.527324 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527297 2565 server.go:87] "Starting device plugin registration server" Sep 10 17:41:11.527700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527685 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 17:41:11.527748 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527705 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 17:41:11.527906 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527823 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 17:41:11.527963 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527927 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 17:41:11.527963 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.527938 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 17:41:11.528663 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.528640 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 17:41:11.528756 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.528700 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:11.615893 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.615861 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 17:41:11.616078 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.615926 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 17:41:11.616078 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.615956 2565 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 17:41:11.616078 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.616002 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 17:41:11.618483 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.618459 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:41:11.628624 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.628607 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.629523 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.629508 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.629593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.629540 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.629593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.629553 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.629593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.629575 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.637861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.637846 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.637924 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.637869 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-57.ec2.internal\": node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:11.655024 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.654948 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:11.717034 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.716999 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal"] Sep 10 17:41:11.717152 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.717089 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.718103 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.718086 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.718182 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.718116 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.718182 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.718125 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.719400 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.719388 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.719511 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.719497 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.719553 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.719527 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.720097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.720073 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.720097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.720089 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.720264 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.720104 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.720264 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.720110 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.720264 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.720118 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.720264 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.720119 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.721406 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.721390 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.721480 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.721422 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:41:11.722059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.722048 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:41:11.722131 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.722072 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:41:11.722131 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.722085 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:41:11.746866 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.746843 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-57.ec2.internal\" not found" node="ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.751298 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.751281 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-57.ec2.internal\" not found" node="ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.755282 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.755267 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:11.777500 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.777476 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92ea45716af076c0638e62d370678f2e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal\" (UID: \"92ea45716af076c0638e62d370678f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.777590 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.777505 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ea45716af076c0638e62d370678f2e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal\" (UID: \"92ea45716af076c0638e62d370678f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.777590 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.777541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c6e324fa94971b9656e873f1a2a8f1c1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-57.ec2.internal\" (UID: \"c6e324fa94971b9656e873f1a2a8f1c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.856351 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.856317 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:11.878707 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.878678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92ea45716af076c0638e62d370678f2e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal\" (UID: \"92ea45716af076c0638e62d370678f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.878766 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.878710 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ea45716af076c0638e62d370678f2e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal\" (UID: \"92ea45716af076c0638e62d370678f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.878766 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.878732 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c6e324fa94971b9656e873f1a2a8f1c1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-57.ec2.internal\" (UID: \"c6e324fa94971b9656e873f1a2a8f1c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.878829 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.878774 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c6e324fa94971b9656e873f1a2a8f1c1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-57.ec2.internal\" (UID: \"c6e324fa94971b9656e873f1a2a8f1c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.878829 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.878782 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92ea45716af076c0638e62d370678f2e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal\" (UID: \"92ea45716af076c0638e62d370678f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.878829 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:11.878799 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ea45716af076c0638e62d370678f2e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal\" (UID: \"92ea45716af076c0638e62d370678f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:11.957085 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:11.957022 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.048539 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.048494 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:12.054073 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.054053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" Sep 10 17:41:12.057763 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.057741 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.158291 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.158252 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.264007 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.263934 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.364550 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.364513 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.415567 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.415528 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 17:41:12.415971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.415711 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 17:41:12.465576 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.465535 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.483896 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.483854 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 17:36:11 +0000 UTC" deadline="2028-06-08 10:29:51.869839414 +0000 UTC" Sep 10 17:41:12.483896 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.483891 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15280h48m39.385952382s" Sep 10 17:41:12.566167 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.566100 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.610873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.610847 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:41:12.611093 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:12.611072 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e324fa94971b9656e873f1a2a8f1c1.slice/crio-36407c245e5afec29de7dd4d43991af636baac04131649a4476d2853f9b3e2ef WatchSource:0}: Error finding container 36407c245e5afec29de7dd4d43991af636baac04131649a4476d2853f9b3e2ef: Status 404 returned error can't find the container with id 36407c245e5afec29de7dd4d43991af636baac04131649a4476d2853f9b3e2ef Sep 10 17:41:12.611589 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:12.611567 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ea45716af076c0638e62d370678f2e.slice/crio-48712fdbef201b846afc19335a79070370848db87def926eb09387e57f8afc7f WatchSource:0}: Error finding container 48712fdbef201b846afc19335a79070370848db87def926eb09387e57f8afc7f: Status 404 returned error can't find the container with id 48712fdbef201b846afc19335a79070370848db87def926eb09387e57f8afc7f Sep 10 17:41:12.618423 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.618405 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:41:12.619449 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.619408 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" event={"ID":"92ea45716af076c0638e62d370678f2e","Type":"ContainerStarted","Data":"48712fdbef201b846afc19335a79070370848db87def926eb09387e57f8afc7f"} Sep 10 17:41:12.620784 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:12.620763 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" event={"ID":"c6e324fa94971b9656e873f1a2a8f1c1","Type":"ContainerStarted","Data":"36407c245e5afec29de7dd4d43991af636baac04131649a4476d2853f9b3e2ef"} Sep 10 17:41:12.669103 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.669057 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.775432 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.775400 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.875948 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.875862 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:12.976865 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:12.976832 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-57.ec2.internal\" not found" Sep 10 17:41:13.000515 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.000489 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:41:13.032091 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.032060 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:41:13.078017 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.077976 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" Sep 10 17:41:13.092853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.092804 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 17:41:13.094138 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.094102 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" Sep 10 17:41:13.101748 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.101723 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 17:41:13.456666 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.456632 2565 apiserver.go:52] "Watching apiserver" Sep 10 17:41:13.464905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.464877 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 17:41:13.465379 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.465342 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vqz5s","kube-system/konnectivity-agent-6rs8k","kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b","openshift-cluster-node-tuning-operator/tuned-gf7jq","openshift-dns/node-resolver-6b7qr","openshift-image-registry/node-ca-mh7pm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal","openshift-multus/multus-additional-cni-plugins-9wd6j","openshift-multus/multus-zjrg9","openshift-multus/network-metrics-daemon-g2jbw","openshift-network-diagnostics/network-check-target-prwhp","openshift-network-operator/iptables-alerter-cjb2d","openshift-ovn-kubernetes/ovnkube-node-22nhh"] Sep 10 17:41:13.466855 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.466833 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.467943 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.467845 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.470545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.469435 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.470545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.469622 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.470545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.469928 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 17:41:13.470545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.469965 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8gbqd\"" Sep 10 17:41:13.470545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.469965 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.470853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.470793 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.471080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.471012 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlqs7\"" Sep 10 17:41:13.471175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.471080 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 17:41:13.471175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.471095 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.471175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.471120 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.472184 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.472166 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.472556 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.472336 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 17:41:13.472556 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.472400 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.472556 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.472411 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 17:41:13.472750 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.472706 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.473622 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.473604 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 17:41:13.473622 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.473604 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 17:41:13.473759 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.473656 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 17:41:13.473759 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.473657 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7pdqt\"" Sep 10 17:41:13.473858 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.473822 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.474368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.474340 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.474368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.474343 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.474741 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.474714 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kchp5\"" Sep 10 17:41:13.475430 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.474926 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.475910 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.475814 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.475997 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.475983 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 17:41:13.476381 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.476362 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.476582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.476567 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.476656 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.476630 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:13.477142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.477122 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ptr64\"" Sep 10 17:41:13.477753 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.477737 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.477838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.477769 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.477983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.477969 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.478851 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.478833 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 17:41:13.478946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.478853 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-47zt4\"" Sep 10 17:41:13.479474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.479335 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 17:41:13.479682 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.479637 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.481845 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.481824 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 17:41:13.481952 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.481825 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 17:41:13.482087 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.482044 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:13.482607 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.482577 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:13.483981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.483962 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:13.484054 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.484031 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:13.488465 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-ovn\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.488586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488478 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovn-node-metrics-cert\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.488586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488505 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-modprobe-d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.488586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488530 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-tuned\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.488586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488554 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-tmp-dir\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.488586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2cd31ec3-6041-40ea-a2cd-a991b38024c7-kubelet-config\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488601 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2cd31ec3-6041-40ea-a2cd-a991b38024c7-dbus\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc6kw\" (UniqueName: \"kubernetes.io/projected/98733f1c-e61e-48e1-8025-38d5a558d6c8-kube-api-access-wc6kw\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488646 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxk2b\" (UniqueName: \"kubernetes.io/projected/067eb756-a003-43b7-8693-116193f3ae2c-kube-api-access-gxk2b\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlm4x\" (UniqueName: \"kubernetes.io/projected/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-kube-api-access-xlm4x\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-systemd\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488714 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-kubernetes\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488736 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-lib-modules\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488758 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-var-lib-kubelet\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-systemd-units\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.488838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488825 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-socket-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-hosts-file\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488873 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/067eb756-a003-43b7-8693-116193f3ae2c-serviceca\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488906 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovnkube-script-lib\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysconfig\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488949 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-var-lib-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488974 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-etc-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.488999 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-log-socket\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-etc-selinux\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-sys-fs\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489081 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cnfmh\"" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489099 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-kubelet\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-run\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489318 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-slash\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.489361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489345 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5c4q\" (UniqueName: \"kubernetes.io/projected/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-kube-api-access-j5c4q\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-run-netns\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489419 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-cni-bin\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-system-cni-dir\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/067eb756-a003-43b7-8693-116193f3ae2c-host\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489531 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489583 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-cni-netd\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489607 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-systemd\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489635 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-sys\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489658 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4tpvn\"" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489842 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489905 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-cnibin\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.489988 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490013 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-registration-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.490100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490040 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-host\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490062 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-tmp\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490092 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-iptables-alerter-script\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490108 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ksw5r\"" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-node-log\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490139 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovnkube-config\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-env-overrides\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-device-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490216 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcbx9\" (UniqueName: \"kubernetes.io/projected/09e5252b-3e09-479b-bdb0-96f9ed6d770b-kube-api-access-gcbx9\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysctl-conf\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490294 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490320 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-host-slash\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysctl-d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbzcj\" (UniqueName: \"kubernetes.io/projected/fd061ead-6c8f-42ce-b26a-29377ea343a0-kube-api-access-qbzcj\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.490924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.490396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrx6d\" (UniqueName: \"kubernetes.io/projected/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-kube-api-access-hrx6d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.500763 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.500736 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 17:41:13.527285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.527256 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2wq5g" Sep 10 17:41:13.540296 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.540261 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2wq5g" Sep 10 17:41:13.576602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.576576 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 17:41:13.590570 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590538 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysconfig\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-multus-certs\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfzw9\" (UniqueName: \"kubernetes.io/projected/e0aa3af8-0964-41e7-9863-552d00698063-kube-api-access-bfzw9\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wjhj\" (UniqueName: \"kubernetes.io/projected/d9588d65-3514-42ee-9252-56457aeeac79-kube-api-access-6wjhj\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-var-lib-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590652 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-etc-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590667 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-log-socket\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590685 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590701 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-etc-selinux\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590715 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-sys-fs\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590730 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-cnibin\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-kubelet\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.590751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590773 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-run\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590805 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-slash\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.590836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5c4q\" (UniqueName: \"kubernetes.io/projected/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-kube-api-access-j5c4q\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591193 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-etc-selinux\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591217 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-etc-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591209 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysconfig\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591245 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-hostroot\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591285 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-log-socket\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-run\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591390 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-sys-fs\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591397 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-var-lib-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-slash\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-run-netns\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591440 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591453 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-kubelet\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591523 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-cni-bin\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591548 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591565 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-run-netns\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591600 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-cni-bin\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-system-cni-dir\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591666 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-system-cni-dir\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591748 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/067eb756-a003-43b7-8693-116193f3ae2c-host\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.591959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591789 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-etc-kubernetes\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591811 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-cni-netd\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591858 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-openvswitch\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591895 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/067eb756-a003-43b7-8693-116193f3ae2c-host\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591913 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-systemd\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591947 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-host-cni-netd\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591964 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591969 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-sys\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.591986 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-systemd\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592024 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-cnibin\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592033 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-sys\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cbc90ac2-79a2-42fe-8212-55b616354d80-konnectivity-ca\") pod \"konnectivity-agent-6rs8k\" (UID: \"cbc90ac2-79a2-42fe-8212-55b616354d80\") " pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd061ead-6c8f-42ce-b26a-29377ea343a0-cnibin\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-kubelet\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.592555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-registration-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-host\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-tmp\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592283 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-host\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592292 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-registration-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0aa3af8-0964-41e7-9863-552d00698063-multus-daemon-config\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592360 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-iptables-alerter-script\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592392 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0aa3af8-0964-41e7-9863-552d00698063-cni-binary-copy\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-k8s-cni-cncf-io\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-node-log\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovnkube-config\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592492 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-node-log\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-env-overrides\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-device-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcbx9\" (UniqueName: \"kubernetes.io/projected/09e5252b-3e09-479b-bdb0-96f9ed6d770b-kube-api-access-gcbx9\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592584 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysctl-conf\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.593376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-cni-bin\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-host-slash\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysctl-d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbzcj\" (UniqueName: \"kubernetes.io/projected/fd061ead-6c8f-42ce-b26a-29377ea343a0-kube-api-access-qbzcj\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592725 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-cni-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592745 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-socket-dir-parent\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592768 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrx6d\" (UniqueName: \"kubernetes.io/projected/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-kube-api-access-hrx6d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-cni-multus\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-ovn\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-iptables-alerter-script\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovn-node-metrics-cert\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592860 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-modprobe-d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592882 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysctl-conf\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-tuned\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592912 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-device-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.594158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-tmp-dir\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.592944 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-env-overrides\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593007 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-ovn\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2cd31ec3-6041-40ea-a2cd-a991b38024c7-kubelet-config\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593064 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2cd31ec3-6041-40ea-a2cd-a991b38024c7-dbus\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc6kw\" (UniqueName: \"kubernetes.io/projected/98733f1c-e61e-48e1-8025-38d5a558d6c8-kube-api-access-wc6kw\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxk2b\" (UniqueName: \"kubernetes.io/projected/067eb756-a003-43b7-8693-116193f3ae2c-kube-api-access-gxk2b\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlm4x\" (UniqueName: \"kubernetes.io/projected/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-kube-api-access-xlm4x\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593161 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-sysctl-d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-systemd\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593219 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-run-systemd\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593101 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-host-slash\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd061ead-6c8f-42ce-b26a-29377ea343a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593431 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2cd31ec3-6041-40ea-a2cd-a991b38024c7-kubelet-config\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593491 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2cd31ec3-6041-40ea-a2cd-a991b38024c7-dbus\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593505 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-kubernetes\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-lib-modules\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595005 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-var-lib-kubelet\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593584 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593609 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-tmp-dir\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593627 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-kubernetes\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-system-cni-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-modprobe-d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593820 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-lib-modules\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.593839 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-systemd-units\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-socket-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.593906 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:14.093887568 +0000 UTC m=+2.975612716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-hosts-file\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593947 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/067eb756-a003-43b7-8693-116193f3ae2c-serviceca\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593968 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09e5252b-3e09-479b-bdb0-96f9ed6d770b-socket-dir\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.593969 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cbc90ac2-79a2-42fe-8212-55b616354d80-agent-certs\") pod \"konnectivity-agent-6rs8k\" (UID: \"cbc90ac2-79a2-42fe-8212-55b616354d80\") " pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-var-lib-kubelet\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594041 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovnkube-config\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.595811 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98733f1c-e61e-48e1-8025-38d5a558d6c8-systemd-units\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594130 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-netns\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-conf-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594166 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-hosts-file\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594205 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovnkube-script-lib\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594436 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/067eb756-a003-43b7-8693-116193f3ae2c-serviceca\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.594715 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovnkube-script-lib\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.596671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.596665 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-etc-tuned\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.596904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.596720 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-tmp\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.596904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.596778 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98733f1c-e61e-48e1-8025-38d5a558d6c8-ovn-node-metrics-cert\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.606036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.602736 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5c4q\" (UniqueName: \"kubernetes.io/projected/79157419-0d17-4cb0-8ebe-e1efbb0fd39c-kube-api-access-j5c4q\") pod \"node-resolver-6b7qr\" (UID: \"79157419-0d17-4cb0-8ebe-e1efbb0fd39c\") " pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.606036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.603250 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcbx9\" (UniqueName: \"kubernetes.io/projected/09e5252b-3e09-479b-bdb0-96f9ed6d770b-kube-api-access-gcbx9\") pod \"aws-ebs-csi-driver-node-xqc5b\" (UID: \"09e5252b-3e09-479b-bdb0-96f9ed6d770b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.606036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.604588 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbzcj\" (UniqueName: \"kubernetes.io/projected/fd061ead-6c8f-42ce-b26a-29377ea343a0-kube-api-access-qbzcj\") pod \"multus-additional-cni-plugins-9wd6j\" (UID: \"fd061ead-6c8f-42ce-b26a-29377ea343a0\") " pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.608785 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.608523 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc6kw\" (UniqueName: \"kubernetes.io/projected/98733f1c-e61e-48e1-8025-38d5a558d6c8-kube-api-access-wc6kw\") pod \"ovnkube-node-22nhh\" (UID: \"98733f1c-e61e-48e1-8025-38d5a558d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.608785 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.608709 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrx6d\" (UniqueName: \"kubernetes.io/projected/bdadf91d-81b4-4d55-b2f3-1d52029e55e0-kube-api-access-hrx6d\") pod \"tuned-gf7jq\" (UID: \"bdadf91d-81b4-4d55-b2f3-1d52029e55e0\") " pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.608996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.608944 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlm4x\" (UniqueName: \"kubernetes.io/projected/cf1606c1-9a9c-4fc8-a431-2bbade2c9611-kube-api-access-xlm4x\") pod \"iptables-alerter-cjb2d\" (UID: \"cf1606c1-9a9c-4fc8-a431-2bbade2c9611\") " pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.609132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.609095 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxk2b\" (UniqueName: \"kubernetes.io/projected/067eb756-a003-43b7-8693-116193f3ae2c-kube-api-access-gxk2b\") pod \"node-ca-mh7pm\" (UID: \"067eb756-a003-43b7-8693-116193f3ae2c\") " pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.653425 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.653397 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:41:13.695333 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cbc90ac2-79a2-42fe-8212-55b616354d80-konnectivity-ca\") pod \"konnectivity-agent-6rs8k\" (UID: \"cbc90ac2-79a2-42fe-8212-55b616354d80\") " pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695342 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-kubelet\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0aa3af8-0964-41e7-9863-552d00698063-multus-daemon-config\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695389 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0aa3af8-0964-41e7-9863-552d00698063-cni-binary-copy\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695435 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-k8s-cni-cncf-io\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-cni-bin\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695469 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-kubelet\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-cni-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-socket-dir-parent\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695536 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-k8s-cni-cncf-io\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-cni-multus\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-system-cni-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695617 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cbc90ac2-79a2-42fe-8212-55b616354d80-agent-certs\") pod \"konnectivity-agent-6rs8k\" (UID: \"cbc90ac2-79a2-42fe-8212-55b616354d80\") " pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.695627 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-netns\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695651 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-conf-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.695683 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:14.195661986 +0000 UTC m=+3.077387119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695705 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-conf-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-multus-certs\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695742 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfzw9\" (UniqueName: \"kubernetes.io/projected/e0aa3af8-0964-41e7-9863-552d00698063-kube-api-access-bfzw9\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695768 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wjhj\" (UniqueName: \"kubernetes.io/projected/d9588d65-3514-42ee-9252-56457aeeac79-kube-api-access-6wjhj\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695784 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-cni-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-cnibin\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695825 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-multus-socket-dir-parent\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.695894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695829 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695852 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-cni-multus\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695855 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-hostroot\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-etc-kubernetes\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695894 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cbc90ac2-79a2-42fe-8212-55b616354d80-konnectivity-ca\") pod \"konnectivity-agent-6rs8k\" (UID: \"cbc90ac2-79a2-42fe-8212-55b616354d80\") " pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695988 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0aa3af8-0964-41e7-9863-552d00698063-multus-daemon-config\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696054 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-netns\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.695886 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-system-cni-dir\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696118 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-cnibin\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696156 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-run-multus-certs\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696195 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-host-var-lib-cni-bin\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696282 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-etc-kubernetes\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696341 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0aa3af8-0964-41e7-9863-552d00698063-hostroot\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.696675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.696370 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0aa3af8-0964-41e7-9863-552d00698063-cni-binary-copy\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.698494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.698472 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cbc90ac2-79a2-42fe-8212-55b616354d80-agent-certs\") pod \"konnectivity-agent-6rs8k\" (UID: \"cbc90ac2-79a2-42fe-8212-55b616354d80\") " pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.706437 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.706409 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:13.706437 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.706433 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:13.706610 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.706447 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:13.706610 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:13.706506 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:14.206487779 +0000 UTC m=+3.088212919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:13.708460 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.708412 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfzw9\" (UniqueName: \"kubernetes.io/projected/e0aa3af8-0964-41e7-9863-552d00698063-kube-api-access-bfzw9\") pod \"multus-zjrg9\" (UID: \"e0aa3af8-0964-41e7-9863-552d00698063\") " pod="openshift-multus/multus-zjrg9" Sep 10 17:41:13.708786 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.708765 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wjhj\" (UniqueName: \"kubernetes.io/projected/d9588d65-3514-42ee-9252-56457aeeac79-kube-api-access-6wjhj\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:13.779803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.779764 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mh7pm" Sep 10 17:41:13.791140 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.791115 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cjb2d" Sep 10 17:41:13.801033 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.801006 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:13.806830 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.806811 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" Sep 10 17:41:13.814399 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.814376 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" Sep 10 17:41:13.821946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.821922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6b7qr" Sep 10 17:41:13.830391 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.830369 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" Sep 10 17:41:13.838939 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.838922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:13.845417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:13.845389 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zjrg9" Sep 10 17:41:14.098492 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.098407 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:14.098646 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.098574 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:14.098712 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.098649 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:15.098626389 +0000 UTC m=+3.980351517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:14.198876 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.198843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:14.199061 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.198980 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:14.199061 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.199049 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:15.199030956 +0000 UTC m=+4.080756076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:14.300033 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.300003 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:14.300205 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.300161 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:14.300205 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.300177 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:14.300205 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.300190 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:14.300367 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.300262 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:15.300242823 +0000 UTC m=+4.181967962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:14.543148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.543072 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:36:13 +0000 UTC" deadline="2028-04-19 22:10:59.038201578 +0000 UTC" Sep 10 17:41:14.543148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.543104 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14092h29m44.495102414s" Sep 10 17:41:14.558808 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:14.558668 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1606c1_9a9c_4fc8_a431_2bbade2c9611.slice/crio-83d7e81f753a377766863abd7892d8ee51966d8a17563868fdfdb8865e3582e3 WatchSource:0}: Error finding container 83d7e81f753a377766863abd7892d8ee51966d8a17563868fdfdb8865e3582e3: Status 404 returned error can't find the container with id 83d7e81f753a377766863abd7892d8ee51966d8a17563868fdfdb8865e3582e3 Sep 10 17:41:14.564149 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:14.564112 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd061ead_6c8f_42ce_b26a_29377ea343a0.slice/crio-c4540cb3596c9e353e41e24b51adafd58577dca0aabbb0f29044c3278d70828f WatchSource:0}: Error finding container c4540cb3596c9e353e41e24b51adafd58577dca0aabbb0f29044c3278d70828f: Status 404 returned error can't find the container with id c4540cb3596c9e353e41e24b51adafd58577dca0aabbb0f29044c3278d70828f Sep 10 17:41:14.567791 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:14.567769 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdadf91d_81b4_4d55_b2f3_1d52029e55e0.slice/crio-e753a5846bb58dfdc3913cab9303281538a771d1f918f44d4e9b37e40ec64ac9 WatchSource:0}: Error finding container e753a5846bb58dfdc3913cab9303281538a771d1f918f44d4e9b37e40ec64ac9: Status 404 returned error can't find the container with id e753a5846bb58dfdc3913cab9303281538a771d1f918f44d4e9b37e40ec64ac9 Sep 10 17:41:14.572581 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:14.571634 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98733f1c_e61e_48e1_8025_38d5a558d6c8.slice/crio-6898aa0957a032578abf98e5e4f25ecfe5f1b62d7f2697c89abf2804d414a7e0 WatchSource:0}: Error finding container 6898aa0957a032578abf98e5e4f25ecfe5f1b62d7f2697c89abf2804d414a7e0: Status 404 returned error can't find the container with id 6898aa0957a032578abf98e5e4f25ecfe5f1b62d7f2697c89abf2804d414a7e0 Sep 10 17:41:14.616738 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.616715 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:14.616845 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:14.616823 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:14.625837 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.625812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" event={"ID":"bdadf91d-81b4-4d55-b2f3-1d52029e55e0","Type":"ContainerStarted","Data":"e753a5846bb58dfdc3913cab9303281538a771d1f918f44d4e9b37e40ec64ac9"} Sep 10 17:41:14.627273 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.627251 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mh7pm" event={"ID":"067eb756-a003-43b7-8693-116193f3ae2c","Type":"ContainerStarted","Data":"844d30edad5acd947706ed36fce8d7ca51c38a652fc35ac458112844a0a38e1a"} Sep 10 17:41:14.628400 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.628380 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"6898aa0957a032578abf98e5e4f25ecfe5f1b62d7f2697c89abf2804d414a7e0"} Sep 10 17:41:14.629553 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.629533 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zjrg9" event={"ID":"e0aa3af8-0964-41e7-9863-552d00698063","Type":"ContainerStarted","Data":"c5b41c5cef73e2da0b8b34716c9aaf9945f6dc227977cd65b3095826cc9bef06"} Sep 10 17:41:14.630521 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.630497 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6b7qr" event={"ID":"79157419-0d17-4cb0-8ebe-e1efbb0fd39c","Type":"ContainerStarted","Data":"1fde02a50eee83fbdef23c4c590a2037797492924862d7f61de8b6ef65e97334"} Sep 10 17:41:14.631569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.631543 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6rs8k" event={"ID":"cbc90ac2-79a2-42fe-8212-55b616354d80","Type":"ContainerStarted","Data":"601c0a47864141087c89c72a15cd4f794309039853e7464a56352821ecddb480"} Sep 10 17:41:14.632900 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.632770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" event={"ID":"09e5252b-3e09-479b-bdb0-96f9ed6d770b","Type":"ContainerStarted","Data":"a3e38dd1bcefca2aafda9091b8954a33c86d1cce2456cc55dd52e5f6f79749d1"} Sep 10 17:41:14.633988 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.633967 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerStarted","Data":"c4540cb3596c9e353e41e24b51adafd58577dca0aabbb0f29044c3278d70828f"} Sep 10 17:41:14.635147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:14.635127 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cjb2d" event={"ID":"cf1606c1-9a9c-4fc8-a431-2bbade2c9611","Type":"ContainerStarted","Data":"83d7e81f753a377766863abd7892d8ee51966d8a17563868fdfdb8865e3582e3"} Sep 10 17:41:15.106884 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.106801 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:15.107046 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.106979 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:15.107046 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.107044 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:17.107024276 +0000 UTC m=+5.988749398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:15.207442 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.207404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:15.207619 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.207590 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:15.207719 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.207704 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:17.207665089 +0000 UTC m=+6.089390209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:15.307962 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.307927 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:15.308163 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.308147 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:15.308247 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.308170 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:15.308247 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.308184 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:15.308368 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.308255 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:17.308236373 +0000 UTC m=+6.189961508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:15.543700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.543594 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:36:13 +0000 UTC" deadline="2028-05-13 15:14:06.82755474 +0000 UTC" Sep 10 17:41:15.543700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.543652 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14661h32m51.283906827s" Sep 10 17:41:15.617365 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.617340 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:15.617523 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.617476 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:15.619736 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.619713 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:15.619847 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:15.619819 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:15.653110 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.653078 2565 generic.go:356] "Generic (PLEG): container finished" podID="92ea45716af076c0638e62d370678f2e" containerID="2823937859dd1c4802f4ddb668897d345a9c33f7a636ea9d5a9b43d46e8e03fd" exitCode=0 Sep 10 17:41:15.653367 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.653343 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" event={"ID":"92ea45716af076c0638e62d370678f2e","Type":"ContainerDied","Data":"2823937859dd1c4802f4ddb668897d345a9c33f7a636ea9d5a9b43d46e8e03fd"} Sep 10 17:41:15.659522 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:15.659493 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" event={"ID":"c6e324fa94971b9656e873f1a2a8f1c1","Type":"ContainerStarted","Data":"5ac3b3ea8544dbabbe11a3330ef5ade62308eb3d9032d6821b01c7c7cc56f4d4"} Sep 10 17:41:16.617140 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:16.616653 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:16.617140 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:16.616770 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:16.688984 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:16.688900 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" event={"ID":"92ea45716af076c0638e62d370678f2e","Type":"ContainerStarted","Data":"33d792199ab1efb146281384e276c2e6afd3dc754dec77ba776e24c45b25730e"} Sep 10 17:41:16.707011 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:16.705614 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-57.ec2.internal" podStartSLOduration=3.705592776 podStartE2EDuration="3.705592776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:15.686015532 +0000 UTC m=+4.567740673" watchObservedRunningTime="2026-09-10 17:41:16.705592776 +0000 UTC m=+5.587317918" Sep 10 17:41:17.123461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:17.123417 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:17.123631 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.123553 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:17.123631 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.123614 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:21.123596696 +0000 UTC m=+10.005321833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:17.224968 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:17.224319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:17.224968 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.224525 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:17.224968 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.224603 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:21.22458415 +0000 UTC m=+10.106309274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:17.326122 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:17.325501 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:17.326122 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.325657 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:17.326122 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.325679 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:17.326122 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.325693 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:17.326122 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.325750 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:21.325731736 +0000 UTC m=+10.207456855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:17.617983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:17.617348 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:17.617983 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.617473 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:17.617983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:17.617349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:17.617983 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:17.617892 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:18.617158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:18.617130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:18.617358 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:18.617287 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:19.616731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:19.616614 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:19.616731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:19.616649 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:19.617242 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:19.616753 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:19.617310 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:19.617213 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:20.617124 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:20.617041 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:20.617602 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:20.617166 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:21.154924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:21.154880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:21.155097 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.155057 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:21.155170 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.155119 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:29.155100213 +0000 UTC m=+18.036825349 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:21.255580 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:21.255545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:21.255766 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.255680 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:21.255766 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.255747 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:29.255728111 +0000 UTC m=+18.137453249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:21.356156 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:21.356089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:21.356336 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.356253 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:21.356336 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.356275 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:21.356336 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.356286 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:21.356496 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.356347 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:29.356328904 +0000 UTC m=+18.238054042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:21.617423 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:21.617349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:21.617836 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.617472 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:21.617836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:21.617356 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:21.617957 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:21.617903 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:22.616615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:22.616584 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:22.616786 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:22.616713 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:23.616659 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:23.616473 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:23.617132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:23.616548 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:23.617132 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:23.616740 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:23.617132 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:23.616837 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:24.617164 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:24.617135 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:24.617491 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:24.617273 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:25.616650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:25.616614 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:25.616817 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:25.616753 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:25.616886 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:25.616813 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:25.616966 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:25.616939 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:26.616657 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:26.616619 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:26.617083 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:26.616742 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:27.616265 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:27.616220 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:27.616446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:27.616358 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:27.616446 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:27.616359 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:27.616552 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:27.616475 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:28.616237 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:28.616142 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:28.616642 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:28.616262 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:29.211445 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:29.211412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:29.211609 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.211566 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:29.211664 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.211641 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:45.211620726 +0000 UTC m=+34.093345844 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:29.312182 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:29.312145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:29.312344 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.312303 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:29.312383 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.312370 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:45.312349988 +0000 UTC m=+34.194075123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:29.412630 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:29.412590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:29.412788 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.412734 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:29.412788 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.412755 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:29.412788 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.412768 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:29.412955 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.412829 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:45.412810483 +0000 UTC m=+34.294535619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:29.616814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:29.616715 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:29.616814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:29.616726 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:29.617294 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.616838 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:29.617294 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:29.616979 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:30.616558 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:30.616523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:30.616736 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:30.616626 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:31.617551 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:31.617513 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:31.617887 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:31.617588 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:31.617887 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:31.617693 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:31.617887 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:31.617813 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:32.617058 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.616895 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:32.617167 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:32.617125 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:32.717936 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.717870 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd061ead-6c8f-42ce-b26a-29377ea343a0" containerID="8c29d238b856607ccee4926ea94edecc9527e7851422c24042d1177fbbb90cf4" exitCode=0 Sep 10 17:41:32.718633 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.717965 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerDied","Data":"8c29d238b856607ccee4926ea94edecc9527e7851422c24042d1177fbbb90cf4"} Sep 10 17:41:32.719794 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.719774 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" event={"ID":"bdadf91d-81b4-4d55-b2f3-1d52029e55e0","Type":"ContainerStarted","Data":"4466407fb4a5833f58f328d47003a76b40c848114157edbe3f528697da7c9c78"} Sep 10 17:41:32.721491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.721467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mh7pm" event={"ID":"067eb756-a003-43b7-8693-116193f3ae2c","Type":"ContainerStarted","Data":"bf977562bcc5b3ab7489e097ab47737ef182157580fb86472f2968e6a7a44ae4"} Sep 10 17:41:32.724921 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.724897 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"b1b1f731b8c98e33d93b4b1601d1c3bcfadab0400b9f5a1a58e40ce73ad93c5d"} Sep 10 17:41:32.725027 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.724965 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"d6eb13c67e490c92bb55162f0a5f00cbd6b730a93513ddf0ae291f82f56db7d1"} Sep 10 17:41:32.725027 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.725007 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"5ec29e44169dab16ac40b0e636122278c160a60c9580aa08b6a7eaf63bdcc740"} Sep 10 17:41:32.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.725038 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"9a7452e70bde6eb480d7b30628ba35c79d1ba183f8460556f782871ef5f262df"} Sep 10 17:41:32.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.725067 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"8b3ea61a2484c3930cc0216aff80ec9b5d9d05b8e24061e26cd20384e9899477"} Sep 10 17:41:32.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.725098 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"c6c87e6a84954f75384699ec48d3c5ba48a57a0db5a3bd7c2acbd67711dab9d9"} Sep 10 17:41:32.726461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.726419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zjrg9" event={"ID":"e0aa3af8-0964-41e7-9863-552d00698063","Type":"ContainerStarted","Data":"5dce49727b6d0a2a2c30b38720b2bb146cc8bae5c9f4022dfe41436289a70559"} Sep 10 17:41:32.728003 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.727981 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6b7qr" event={"ID":"79157419-0d17-4cb0-8ebe-e1efbb0fd39c","Type":"ContainerStarted","Data":"898691f1bc83257467bcff765408a0e6af56d84679f687ff47dd9289babb92c1"} Sep 10 17:41:32.729254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.729220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6rs8k" event={"ID":"cbc90ac2-79a2-42fe-8212-55b616354d80","Type":"ContainerStarted","Data":"ff28cf1262a6c7135347f700a16e35ef9e6af0b192e3cd718283cd25186046de"} Sep 10 17:41:32.730668 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.730651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" event={"ID":"09e5252b-3e09-479b-bdb0-96f9ed6d770b","Type":"ContainerStarted","Data":"8d03838df933292c5e7eb93fdd0bb0c2a2acb37c933c53119e7d85d19c18d655"} Sep 10 17:41:32.744357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.744320 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-57.ec2.internal" podStartSLOduration=19.744307911 podStartE2EDuration="19.744307911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:16.706897221 +0000 UTC m=+5.588622365" watchObservedRunningTime="2026-09-10 17:41:32.744307911 +0000 UTC m=+21.626033052" Sep 10 17:41:32.760682 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.760640 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6rs8k" podStartSLOduration=8.086867436 podStartE2EDuration="21.76062325s" totalImagesPullingTime="13.673755814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:32.76059247 +0000 UTC m=+21.642317612" watchObservedRunningTime="2026-09-10 17:41:32.76062325 +0000 UTC m=+21.642348392" Sep 10 17:41:32.778514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.778476 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mh7pm" podStartSLOduration=4.421465033 podStartE2EDuration="21.778468644s" totalImagesPullingTime="17.357003611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:32.778079282 +0000 UTC m=+21.659804435" watchObservedRunningTime="2026-09-10 17:41:32.778468644 +0000 UTC m=+21.660193785" Sep 10 17:41:32.796014 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.795981 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gf7jq" podStartSLOduration=4.43281573 podStartE2EDuration="21.795971743s" totalImagesPullingTime="17.363156013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:32.795339178 +0000 UTC m=+21.677064319" watchObservedRunningTime="2026-09-10 17:41:32.795971743 +0000 UTC m=+21.677696883" Sep 10 17:41:32.832545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.832299 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6b7qr" podStartSLOduration=4.471567513 podStartE2EDuration="21.832284926s" totalImagesPullingTime="17.360717413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:32.813432891 +0000 UTC m=+21.695158032" watchObservedRunningTime="2026-09-10 17:41:32.832284926 +0000 UTC m=+21.714010067" Sep 10 17:41:32.833008 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:32.832885 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zjrg9" podStartSLOduration=4.457207952 podStartE2EDuration="21.832867355s" totalImagesPullingTime="17.375659403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:32.832088074 +0000 UTC m=+21.713813217" watchObservedRunningTime="2026-09-10 17:41:32.832867355 +0000 UTC m=+21.714592495" Sep 10 17:41:33.469388 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.469182 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 17:41:33.541001 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.540902 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T17:41:33.469385019Z","UUID":"1750402a-5d1b-4c0a-b04a-c23bf204d348","Handler":null,"Name":"","Endpoint":""} Sep 10 17:41:33.542275 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.542254 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 17:41:33.542399 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.542287 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 17:41:33.616208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.616184 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:33.616349 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.616183 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:33.616349 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:33.616294 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:33.616451 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:33.616371 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:33.734974 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.734944 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cjb2d" event={"ID":"cf1606c1-9a9c-4fc8-a431-2bbade2c9611","Type":"ContainerStarted","Data":"43d7ea6a5a04404974fec0e8f494caa34057ee4588a2bb2eca53ce96cc7dedc1"} Sep 10 17:41:33.736998 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:33.736957 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" event={"ID":"09e5252b-3e09-479b-bdb0-96f9ed6d770b","Type":"ContainerStarted","Data":"817641e12182d4c7ee18300abdd1ebf5672b96c9c187059507fa25ad50c61525"} Sep 10 17:41:34.616325 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:34.616301 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:34.616424 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:34.616392 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:34.742252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:34.742153 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"f28609020fa40f75399f6e831a737c98610f5e52ecfb62d35f18f5ab98edb4a7"} Sep 10 17:41:34.744114 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:34.744083 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" event={"ID":"09e5252b-3e09-479b-bdb0-96f9ed6d770b","Type":"ContainerStarted","Data":"de439a055903a097d61316a99ed6f5628ba2e7938f365f433b444f0ecfdab5f2"} Sep 10 17:41:34.767095 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:34.767046 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xqc5b" podStartSLOduration=4.287044704 podStartE2EDuration="23.767032513s" totalImagesPullingTime="19.479987809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:34.766803325 +0000 UTC m=+23.648528465" watchObservedRunningTime="2026-09-10 17:41:34.767032513 +0000 UTC m=+23.648757653" Sep 10 17:41:34.767552 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:34.767521 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cjb2d" podStartSLOduration=10.087425308 podStartE2EDuration="23.767511302s" totalImagesPullingTime="13.680085994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:33.752204275 +0000 UTC m=+22.633929415" watchObservedRunningTime="2026-09-10 17:41:34.767511302 +0000 UTC m=+23.649236443" Sep 10 17:41:35.387471 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:35.387439 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:35.388059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:35.388035 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:35.616875 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:35.616844 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:35.617042 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:35.616853 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:35.617042 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:35.616964 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:35.617159 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:35.617058 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:35.746367 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:35.746267 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:35.746769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:35.746702 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6rs8k" Sep 10 17:41:36.616975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:36.616949 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:36.617154 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:36.617052 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:37.616260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:37.616146 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:37.616260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:37.616153 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:37.616680 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:37.616253 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:37.616680 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:37.616324 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:37.753490 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:37.753456 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" event={"ID":"98733f1c-e61e-48e1-8025-38d5a558d6c8","Type":"ContainerStarted","Data":"94e7279a0d3e32b0a37ac3093004edc39dcd81669ab6eaa5723dafd1c856f749"} Sep 10 17:41:37.753734 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:37.753714 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:37.768406 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:37.768389 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:37.830394 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:37.830188 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" podStartSLOduration=9.279503941 podStartE2EDuration="26.830173866s" totalImagesPullingTime="17.550669925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:37.794611772 +0000 UTC m=+26.676336934" watchObservedRunningTime="2026-09-10 17:41:37.830173866 +0000 UTC m=+26.711899006" Sep 10 17:41:38.616341 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:38.616315 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:38.616763 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:38.616412 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:38.757672 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:38.757640 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd061ead-6c8f-42ce-b26a-29377ea343a0" containerID="348c8e6d1d1e1ec56918be8507f07b6a2d6f0bd124be63b099e6df515f0c779f" exitCode=0 Sep 10 17:41:38.757830 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:38.757731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerDied","Data":"348c8e6d1d1e1ec56918be8507f07b6a2d6f0bd124be63b099e6df515f0c779f"} Sep 10 17:41:38.758151 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:38.758134 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:38.758236 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:38.758211 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:38.772762 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:38.772740 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:41:39.562180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.562152 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vqz5s"] Sep 10 17:41:39.562347 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.562297 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:39.562419 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:39.562400 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:39.565912 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.565888 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2jbw"] Sep 10 17:41:39.566001 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.565990 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:39.566094 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:39.566078 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:39.568350 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.568327 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-prwhp"] Sep 10 17:41:39.568448 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.568413 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:39.568525 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:39.568505 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:39.762469 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.762389 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd061ead-6c8f-42ce-b26a-29377ea343a0" containerID="aac2a6b0bbc63123ceed2b7cef3678e2d0a3a81613610081b4a0df4aa3b4fd40" exitCode=0 Sep 10 17:41:39.762469 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:39.762433 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerDied","Data":"aac2a6b0bbc63123ceed2b7cef3678e2d0a3a81613610081b4a0df4aa3b4fd40"} Sep 10 17:41:40.767367 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:40.767178 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd061ead-6c8f-42ce-b26a-29377ea343a0" containerID="5097c19918a959315adade850992a88fa7c23c70acc02f320e269fb4863be684" exitCode=0 Sep 10 17:41:40.767367 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:40.767256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerDied","Data":"5097c19918a959315adade850992a88fa7c23c70acc02f320e269fb4863be684"} Sep 10 17:41:41.617825 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:41.617721 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:41.617825 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:41.617765 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:41.618039 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:41.617850 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:41.618039 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:41.617886 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:41.618039 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:41.617930 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:41.618039 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:41.618004 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:43.616312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:43.616279 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:43.616312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:43.616308 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:43.616814 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:43.616443 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2jbw" podUID="d9588d65-3514-42ee-9252-56457aeeac79" Sep 10 17:41:43.616814 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:43.616570 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prwhp" podUID="ac66f28e-0601-4dcc-99d7-2a298bc5d51f" Sep 10 17:41:43.616814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:43.616591 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:43.616814 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:43.616680 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vqz5s" podUID="2cd31ec3-6041-40ea-a2cd-a991b38024c7" Sep 10 17:41:44.162105 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.162067 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:44.239370 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.239335 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:44.315627 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.315593 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:44.366401 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.366371 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:44.871756 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.871681 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-57.ec2.internal" event="NodeReady" Sep 10 17:41:44.872154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.871811 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 17:41:44.919613 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.919585 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6tl2g"] Sep 10 17:41:44.953880 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.953854 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5tvgn"] Sep 10 17:41:44.954024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.953977 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:44.956737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.956715 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 17:41:44.956891 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.956720 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 17:41:44.957156 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.957135 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d7j7h\"" Sep 10 17:41:44.967302 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.967282 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6tl2g"] Sep 10 17:41:44.967387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.967324 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5tvgn"] Sep 10 17:41:44.967387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.967382 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:44.970189 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.970165 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 17:41:44.973149 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.970744 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p79lb\"" Sep 10 17:41:44.973149 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.970761 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 17:41:44.973603 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:44.973471 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 17:41:45.030132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.030102 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-config-volume\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.030298 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.030144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.030298 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.030170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-tmp-dir\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.030298 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.030199 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gcfw\" (UniqueName: \"kubernetes.io/projected/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-kube-api-access-7gcfw\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.131477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131393 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94aed250-dd5b-4c9c-aeb5-9178dde79af0-cert\") pod \"ingress-canary-5tvgn\" (UID: \"94aed250-dd5b-4c9c-aeb5-9178dde79af0\") " pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.131477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-config-volume\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.131700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.131700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131517 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-tmp-dir\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.131700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gcfw\" (UniqueName: \"kubernetes.io/projected/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-kube-api-access-7gcfw\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.131700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131575 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2wmf\" (UniqueName: \"kubernetes.io/projected/94aed250-dd5b-4c9c-aeb5-9178dde79af0-kube-api-access-d2wmf\") pod \"ingress-canary-5tvgn\" (UID: \"94aed250-dd5b-4c9c-aeb5-9178dde79af0\") " pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.131700 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.131617 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:45.131700 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.131679 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls podName:a1cde6b8-5c75-434a-8880-1db69cfd7ebe nodeName:}" failed. No retries permitted until 2026-09-10 17:41:45.631659633 +0000 UTC m=+34.513384776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls") pod "dns-default-6tl2g" (UID: "a1cde6b8-5c75-434a-8880-1db69cfd7ebe") : secret "dns-default-metrics-tls" not found Sep 10 17:41:45.131936 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.131845 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-tmp-dir\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.140370 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.140336 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-config-volume\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.145109 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.145082 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gcfw\" (UniqueName: \"kubernetes.io/projected/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-kube-api-access-7gcfw\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.232116 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.232085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94aed250-dd5b-4c9c-aeb5-9178dde79af0-cert\") pod \"ingress-canary-5tvgn\" (UID: \"94aed250-dd5b-4c9c-aeb5-9178dde79af0\") " pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.232352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.232164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2wmf\" (UniqueName: \"kubernetes.io/projected/94aed250-dd5b-4c9c-aeb5-9178dde79af0-kube-api-access-d2wmf\") pod \"ingress-canary-5tvgn\" (UID: \"94aed250-dd5b-4c9c-aeb5-9178dde79af0\") " pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.232352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.232199 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:45.232352 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.232321 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:45.232472 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.232381 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret podName:2cd31ec3-6041-40ea-a2cd-a991b38024c7 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:17.232362158 +0000 UTC m=+66.114087294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret") pod "global-pull-secret-syncer-vqz5s" (UID: "2cd31ec3-6041-40ea-a2cd-a991b38024c7") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:41:45.234750 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.234725 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94aed250-dd5b-4c9c-aeb5-9178dde79af0-cert\") pod \"ingress-canary-5tvgn\" (UID: \"94aed250-dd5b-4c9c-aeb5-9178dde79af0\") " pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.240723 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.240685 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2wmf\" (UniqueName: \"kubernetes.io/projected/94aed250-dd5b-4c9c-aeb5-9178dde79af0-kube-api-access-d2wmf\") pod \"ingress-canary-5tvgn\" (UID: \"94aed250-dd5b-4c9c-aeb5-9178dde79af0\") " pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.278683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.278654 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5tvgn" Sep 10 17:41:45.332771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.332736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:45.332931 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.332859 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:45.332931 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.332918 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs podName:d9588d65-3514-42ee-9252-56457aeeac79 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:17.332901333 +0000 UTC m=+66.214626454 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs") pod "network-metrics-daemon-g2jbw" (UID: "d9588d65-3514-42ee-9252-56457aeeac79") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:41:45.433820 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.433792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:45.433947 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.433916 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:41:45.433947 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.433933 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:41:45.433947 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.433942 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6k954 for pod openshift-network-diagnostics/network-check-target-prwhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:45.434047 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.433993 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954 podName:ac66f28e-0601-4dcc-99d7-2a298bc5d51f nodeName:}" failed. No retries permitted until 2026-09-10 17:42:17.433978495 +0000 UTC m=+66.315703625 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6k954" (UniqueName: "kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954") pod "network-check-target-prwhp" (UID: "ac66f28e-0601-4dcc-99d7-2a298bc5d51f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:41:45.616689 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.616534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:41:45.616835 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.616538 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:41:45.616883 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.616551 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:41:45.619659 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.619636 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 17:41:45.619779 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.619739 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 17:41:45.619928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.619905 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 17:41:45.619928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.619912 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 17:41:45.620313 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.620298 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hkln9\"" Sep 10 17:41:45.620377 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.620361 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fbgqn\"" Sep 10 17:41:45.622438 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.622417 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5tvgn"] Sep 10 17:41:45.626787 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:45.626766 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94aed250_dd5b_4c9c_aeb5_9178dde79af0.slice/crio-5d942628ef6763b3a7902219b987075a9168e624db4e3dd249f5e4aa779f5075 WatchSource:0}: Error finding container 5d942628ef6763b3a7902219b987075a9168e624db4e3dd249f5e4aa779f5075: Status 404 returned error can't find the container with id 5d942628ef6763b3a7902219b987075a9168e624db4e3dd249f5e4aa779f5075 Sep 10 17:41:45.635184 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.635164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:45.635296 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.635280 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:45.635363 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:45.635352 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls podName:a1cde6b8-5c75-434a-8880-1db69cfd7ebe nodeName:}" failed. No retries permitted until 2026-09-10 17:41:46.635329908 +0000 UTC m=+35.517055049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls") pod "dns-default-6tl2g" (UID: "a1cde6b8-5c75-434a-8880-1db69cfd7ebe") : secret "dns-default-metrics-tls" not found Sep 10 17:41:45.781500 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.781473 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5tvgn" event={"ID":"94aed250-dd5b-4c9c-aeb5-9178dde79af0","Type":"ContainerStarted","Data":"5d942628ef6763b3a7902219b987075a9168e624db4e3dd249f5e4aa779f5075"} Sep 10 17:41:45.784676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:45.784653 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerStarted","Data":"68404fa1ce432ded02bf67159c07f9ccdeff501b9116dc64cbe2675e340d0a86"} Sep 10 17:41:46.642808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:46.642769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:46.643404 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:46.642898 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:46.643404 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:46.642977 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls podName:a1cde6b8-5c75-434a-8880-1db69cfd7ebe nodeName:}" failed. No retries permitted until 2026-09-10 17:41:48.642956574 +0000 UTC m=+37.524681708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls") pod "dns-default-6tl2g" (UID: "a1cde6b8-5c75-434a-8880-1db69cfd7ebe") : secret "dns-default-metrics-tls" not found Sep 10 17:41:46.791368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:46.791326 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd061ead-6c8f-42ce-b26a-29377ea343a0" containerID="68404fa1ce432ded02bf67159c07f9ccdeff501b9116dc64cbe2675e340d0a86" exitCode=0 Sep 10 17:41:46.791532 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:46.791394 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerDied","Data":"68404fa1ce432ded02bf67159c07f9ccdeff501b9116dc64cbe2675e340d0a86"} Sep 10 17:41:47.547923 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:47.547886 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:47.798716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:47.798624 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd061ead-6c8f-42ce-b26a-29377ea343a0" containerID="7f05985b8f2c6738488fdb433f5bf790f75cd609f52c90b0554578c2f1079bed" exitCode=0 Sep 10 17:41:47.798716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:47.798681 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerDied","Data":"7f05985b8f2c6738488fdb433f5bf790f75cd609f52c90b0554578c2f1079bed"} Sep 10 17:41:48.102614 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.102574 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:48.157740 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.157718 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:48.448528 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.448495 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv"] Sep 10 17:41:48.457410 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.457388 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.460278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.460258 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 17:41:48.460387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.460280 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 17:41:48.461607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.461586 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:48.461845 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.461823 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-g9bc8\"" Sep 10 17:41:48.462111 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.462093 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:48.462257 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.462218 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv"] Sep 10 17:41:48.557209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.557178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de119af3-a57d-41b9-a334-63c626cc1d27-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.557209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.557211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525k5\" (UniqueName: \"kubernetes.io/projected/de119af3-a57d-41b9-a334-63c626cc1d27-kube-api-access-525k5\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.557411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.557248 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de119af3-a57d-41b9-a334-63c626cc1d27-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.574293 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.574208 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm"] Sep 10 17:41:48.588964 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.588938 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-765484f768-8d5m8"] Sep 10 17:41:48.589082 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.589064 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" Sep 10 17:41:48.592464 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.592439 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kqgwx\"" Sep 10 17:41:48.592702 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.592682 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:48.592803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.592737 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:48.601490 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.601469 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl"] Sep 10 17:41:48.601578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.601561 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.604208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.604189 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g8lbh\"" Sep 10 17:41:48.604459 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.604444 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 17:41:48.604724 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.604706 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 10 17:41:48.606199 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.606152 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 10 17:41:48.606199 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.606186 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 10 17:41:48.606366 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.606186 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 10 17:41:48.606706 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.606692 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 17:41:48.619759 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.619740 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm"] Sep 10 17:41:48.619841 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.619775 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-765484f768-8d5m8"] Sep 10 17:41:48.619841 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.619796 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl"] Sep 10 17:41:48.619918 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.619846 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.623137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.623118 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 17:41:48.623446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.623431 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8k98b\"" Sep 10 17:41:48.623528 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.623462 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 17:41:48.624927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.624904 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 17:41:48.625010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.624954 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 17:41:48.658374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.658333 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de119af3-a57d-41b9-a334-63c626cc1d27-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.658494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.658396 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525k5\" (UniqueName: \"kubernetes.io/projected/de119af3-a57d-41b9-a334-63c626cc1d27-kube-api-access-525k5\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.658494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.658432 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de119af3-a57d-41b9-a334-63c626cc1d27-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.658494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.658487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:48.658707 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.658636 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:48.658767 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.658714 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls podName:a1cde6b8-5c75-434a-8880-1db69cfd7ebe nodeName:}" failed. No retries permitted until 2026-09-10 17:41:52.65869212 +0000 UTC m=+41.540417258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls") pod "dns-default-6tl2g" (UID: "a1cde6b8-5c75-434a-8880-1db69cfd7ebe") : secret "dns-default-metrics-tls" not found Sep 10 17:41:48.659034 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.659008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de119af3-a57d-41b9-a334-63c626cc1d27-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.660119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.660026 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr"] Sep 10 17:41:48.662643 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.662617 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de119af3-a57d-41b9-a334-63c626cc1d27-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.680412 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.680386 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:48.680940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.680920 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr"] Sep 10 17:41:48.683262 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.683245 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-49s64\"" Sep 10 17:41:48.684121 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.684100 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-525k5\" (UniqueName: \"kubernetes.io/projected/de119af3-a57d-41b9-a334-63c626cc1d27-kube-api-access-525k5\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnqlv\" (UID: \"de119af3-a57d-41b9-a334-63c626cc1d27\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.685360 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.685338 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:48.685360 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.685350 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 17:41:48.685496 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.685380 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:48.758789 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.758769 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.758885 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.758816 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5958b543-bd6a-4291-b525-944cc570ea22-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.758927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.758879 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-default-certificate\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.758927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.758907 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.759024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.758958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dng\" (UniqueName: \"kubernetes.io/projected/511026f7-b2e0-47ae-8d79-51eab0beb87a-kube-api-access-r9dng\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.759082 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.759033 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpfh6\" (UniqueName: \"kubernetes.io/projected/b7d62301-70d4-4c47-b170-e7b5dcebae76-kube-api-access-fpfh6\") pod \"volume-data-source-validator-695c895bd6-cc8xm\" (UID: \"b7d62301-70d4-4c47-b170-e7b5dcebae76\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" Sep 10 17:41:48.759135 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.759077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-stats-auth\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.759135 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.759111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kcr9\" (UniqueName: \"kubernetes.io/projected/5958b543-bd6a-4291-b525-944cc570ea22-kube-api-access-4kcr9\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.759335 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.759311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.760132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.760115 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp"] Sep 10 17:41:48.767455 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.767437 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" Sep 10 17:41:48.770355 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.770336 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.771072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.771054 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bcb66bc9-ql8cz"] Sep 10 17:41:48.773659 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.773643 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wkkp9\"" Sep 10 17:41:48.773780 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.773706 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 17:41:48.773981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.773956 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:48.774103 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.774081 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:48.774822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.774807 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 17:41:48.780379 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.780362 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp"] Sep 10 17:41:48.780489 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.780452 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.783191 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.783137 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 17:41:48.783322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.783208 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 17:41:48.783533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.783513 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 17:41:48.783640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.783563 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-stkqn\"" Sep 10 17:41:48.790893 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.790874 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 17:41:48.804803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.804781 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" event={"ID":"fd061ead-6c8f-42ce-b26a-29377ea343a0","Type":"ContainerStarted","Data":"89031dfecd8dd25cef36085a0911261f344881acca7e93da9c0c975f4657e2b4"} Sep 10 17:41:48.808391 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.808371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5tvgn" event={"ID":"94aed250-dd5b-4c9c-aeb5-9178dde79af0","Type":"ContainerStarted","Data":"8d567a065764a859e162bbcd633039a0a604c085672961ac458676ea0e9592b1"} Sep 10 17:41:48.818851 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.818831 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bcb66bc9-ql8cz"] Sep 10 17:41:48.832052 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.832025 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:48.860412 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rnwt\" (UniqueName: \"kubernetes.io/projected/e8ef2482-361b-49bd-8f72-a594191b49fc-kube-api-access-6rnwt\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.860534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kcr9\" (UniqueName: \"kubernetes.io/projected/5958b543-bd6a-4291-b525-944cc570ea22-kube-api-access-4kcr9\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.860534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gm7r\" (UniqueName: \"kubernetes.io/projected/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-kube-api-access-9gm7r\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:48.860534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860543 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.860578 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.860625 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls podName:5958b543-bd6a-4291-b525-944cc570ea22 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.36061191 +0000 UTC m=+38.242337050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzggl" (UID: "5958b543-bd6a-4291-b525-944cc570ea22") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860617 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5958b543-bd6a-4291-b525-944cc570ea22-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860647 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-default-certificate\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.860679 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.360664974 +0000 UTC m=+38.242390118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : configmap references non-existent config key: service-ca.crt Sep 10 17:41:48.860710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dng\" (UniqueName: \"kubernetes.io/projected/511026f7-b2e0-47ae-8d79-51eab0beb87a-kube-api-access-r9dng\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.860726 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860736 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.860762 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.360749151 +0000 UTC m=+38.242474286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : secret "router-metrics-certs-default" not found Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ef2482-361b-49bd-8f72-a594191b49fc-serving-cert\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860821 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8ef2482-361b-49bd-8f72-a594191b49fc-config\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpfh6\" (UniqueName: \"kubernetes.io/projected/b7d62301-70d4-4c47-b170-e7b5dcebae76-kube-api-access-fpfh6\") pod \"volume-data-source-validator-695c895bd6-cc8xm\" (UID: \"b7d62301-70d4-4c47-b170-e7b5dcebae76\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" Sep 10 17:41:48.861238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.860904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-stats-auth\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.861602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.861454 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5958b543-bd6a-4291-b525-944cc570ea22-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.863533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.863512 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-stats-auth\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.863533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.863529 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-default-certificate\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.874905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.874880 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kcr9\" (UniqueName: \"kubernetes.io/projected/5958b543-bd6a-4291-b525-944cc570ea22-kube-api-access-4kcr9\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:48.882072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.882050 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dng\" (UniqueName: \"kubernetes.io/projected/511026f7-b2e0-47ae-8d79-51eab0beb87a-kube-api-access-r9dng\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:48.888814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.888795 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpfh6\" (UniqueName: \"kubernetes.io/projected/b7d62301-70d4-4c47-b170-e7b5dcebae76-kube-api-access-fpfh6\") pod \"volume-data-source-validator-695c895bd6-cc8xm\" (UID: \"b7d62301-70d4-4c47-b170-e7b5dcebae76\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" Sep 10 17:41:48.898693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.898667 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" Sep 10 17:41:48.899444 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.899411 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9wd6j" podStartSLOduration=9.173975063 podStartE2EDuration="37.899387381s" totalImagesPullingTime="28.725412318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:48.899059135 +0000 UTC m=+37.780784279" watchObservedRunningTime="2026-09-10 17:41:48.899387381 +0000 UTC m=+37.781112522" Sep 10 17:41:48.920088 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.920069 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-certificates\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963132 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-image-registry-private-configuration\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-installation-pull-secrets\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-bound-sa-token\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963291 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ef2482-361b-49bd-8f72-a594191b49fc-serving-cert\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.963427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963352 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.964149 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.964065 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:48.964149 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:48.964130 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls podName:b76ec3fb-81ff-4ef8-9854-d1a957f0020c nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.464110192 +0000 UTC m=+38.345835326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rbdfr" (UID: "b76ec3fb-81ff-4ef8-9854-d1a957f0020c") : secret "samples-operator-tls" not found Sep 10 17:41:48.966663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.963386 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rnwt\" (UniqueName: \"kubernetes.io/projected/e8ef2482-361b-49bd-8f72-a594191b49fc-kube-api-access-6rnwt\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.966663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.964412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gm7r\" (UniqueName: \"kubernetes.io/projected/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-kube-api-access-9gm7r\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:48.966663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.964450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-trusted-ca\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.966663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.964578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-ca-trust-extracted\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.966663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.964643 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8ef2482-361b-49bd-8f72-a594191b49fc-config\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.966663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.964667 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkb8c\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-kube-api-access-jkb8c\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:48.967016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.966890 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ef2482-361b-49bd-8f72-a594191b49fc-serving-cert\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.967016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.966916 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8ef2482-361b-49bd-8f72-a594191b49fc-config\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.986718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.986697 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rnwt\" (UniqueName: \"kubernetes.io/projected/e8ef2482-361b-49bd-8f72-a594191b49fc-kube-api-access-6rnwt\") pod \"service-ca-operator-859f94c4b-624fp\" (UID: \"e8ef2482-361b-49bd-8f72-a594191b49fc\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:48.987568 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:48.987546 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gm7r\" (UniqueName: \"kubernetes.io/projected/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-kube-api-access-9gm7r\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:49.067671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-ca-trust-extracted\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.067792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkb8c\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-kube-api-access-jkb8c\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.067792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067700 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-certificates\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.067792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-image-registry-private-configuration\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.067792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-installation-pull-secrets\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.067792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-bound-sa-token\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.068024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.067870 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.068024 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.067961 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:49.068024 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.067976 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcb66bc9-ql8cz: secret "image-registry-tls" not found Sep 10 17:41:49.068024 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.068024 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls podName:f26f85db-bbe5-4cc7-a49e-1eb8268f24e9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.568009041 +0000 UTC m=+38.449734159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls") pod "image-registry-7bcb66bc9-ql8cz" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9") : secret "image-registry-tls" not found Sep 10 17:41:49.068196 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.068020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-trusted-ca\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.071354 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.068090 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-ca-trust-extracted\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.071354 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.068593 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-certificates\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.071354 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.069103 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-trusted-ca\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.071354 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.070817 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-image-registry-private-configuration\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.071354 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.070882 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-installation-pull-secrets\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.073935 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.073893 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5tvgn" podStartSLOduration=2.427486496 podStartE2EDuration="5.073879814s" totalImagesPullingTime="2.646393318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:48.948042551 +0000 UTC m=+37.829767693" watchObservedRunningTime="2026-09-10 17:41:49.073879814 +0000 UTC m=+37.955604954" Sep 10 17:41:49.074072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.074060 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv"] Sep 10 17:41:49.077353 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:49.077274 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde119af3_a57d_41b9_a334_63c626cc1d27.slice/crio-9ded9e64348a72317e59752c2405d48d5ea3249643c0fbdb4c4c6521dd291b40 WatchSource:0}: Error finding container 9ded9e64348a72317e59752c2405d48d5ea3249643c0fbdb4c4c6521dd291b40: Status 404 returned error can't find the container with id 9ded9e64348a72317e59752c2405d48d5ea3249643c0fbdb4c4c6521dd291b40 Sep 10 17:41:49.079470 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.079449 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" Sep 10 17:41:49.080404 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.080381 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-bound-sa-token\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.081606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.081583 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkb8c\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-kube-api-access-jkb8c\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.105760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.105728 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:49.167561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.167455 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:49.189956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.189925 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm"] Sep 10 17:41:49.192640 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:49.192615 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d62301_70d4_4c47_b170_e7b5dcebae76.slice/crio-5a5cdc37f316ce4957332a40809db753fc693a19c43160eb4bc2cadac1a40450 WatchSource:0}: Error finding container 5a5cdc37f316ce4957332a40809db753fc693a19c43160eb4bc2cadac1a40450: Status 404 returned error can't find the container with id 5a5cdc37f316ce4957332a40809db753fc693a19c43160eb4bc2cadac1a40450 Sep 10 17:41:49.227955 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.227926 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:49.370766 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.370712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:49.370857 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.370770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:49.370857 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.370811 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:49.370941 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.370859 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 17:41:49.370941 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.370916 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:50.370896734 +0000 UTC m=+39.252621874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : secret "router-metrics-certs-default" not found Sep 10 17:41:49.370941 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.370918 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:49.371046 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.370970 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls podName:5958b543-bd6a-4291-b525-944cc570ea22 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:50.37095661 +0000 UTC m=+39.252681730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzggl" (UID: "5958b543-bd6a-4291-b525-944cc570ea22") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:49.371046 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.370983 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:50.370977417 +0000 UTC m=+39.252702536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : configmap references non-existent config key: service-ca.crt Sep 10 17:41:49.375299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.375279 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp"] Sep 10 17:41:49.377949 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:41:49.377932 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ef2482_361b_49bd_8f72_a594191b49fc.slice/crio-e1c5b8af9e1fc4ea9f251c232142f7142f1c4cf700f9b4aebe9ca708483d236b WatchSource:0}: Error finding container e1c5b8af9e1fc4ea9f251c232142f7142f1c4cf700f9b4aebe9ca708483d236b: Status 404 returned error can't find the container with id e1c5b8af9e1fc4ea9f251c232142f7142f1c4cf700f9b4aebe9ca708483d236b Sep 10 17:41:49.472074 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.472048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:49.472167 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.472155 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:49.472209 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.472204 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls podName:b76ec3fb-81ff-4ef8-9854-d1a957f0020c nodeName:}" failed. No retries permitted until 2026-09-10 17:41:50.472188988 +0000 UTC m=+39.353914125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rbdfr" (UID: "b76ec3fb-81ff-4ef8-9854-d1a957f0020c") : secret "samples-operator-tls" not found Sep 10 17:41:49.573398 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.573372 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:49.573485 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.573462 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:49.573485 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.573473 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcb66bc9-ql8cz: secret "image-registry-tls" not found Sep 10 17:41:49.573557 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:49.573529 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls podName:f26f85db-bbe5-4cc7-a49e-1eb8268f24e9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:50.573510681 +0000 UTC m=+39.455235816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls") pod "image-registry-7bcb66bc9-ql8cz" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9") : secret "image-registry-tls" not found Sep 10 17:41:49.812467 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.812431 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" event={"ID":"de119af3-a57d-41b9-a334-63c626cc1d27","Type":"ContainerStarted","Data":"9ded9e64348a72317e59752c2405d48d5ea3249643c0fbdb4c4c6521dd291b40"} Sep 10 17:41:49.814042 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.813988 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" event={"ID":"e8ef2482-361b-49bd-8f72-a594191b49fc","Type":"ContainerStarted","Data":"e1c5b8af9e1fc4ea9f251c232142f7142f1c4cf700f9b4aebe9ca708483d236b"} Sep 10 17:41:49.815922 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:49.815896 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" event={"ID":"b7d62301-70d4-4c47-b170-e7b5dcebae76","Type":"ContainerStarted","Data":"5a5cdc37f316ce4957332a40809db753fc693a19c43160eb4bc2cadac1a40450"} Sep 10 17:41:50.382538 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.382496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:50.382712 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.382616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:50.382712 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.382693 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:52.382669355 +0000 UTC m=+41.264394482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : configmap references non-existent config key: service-ca.crt Sep 10 17:41:50.382839 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.382749 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 17:41:50.382839 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.382805 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:52.382788136 +0000 UTC m=+41.264513275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : secret "router-metrics-certs-default" not found Sep 10 17:41:50.382839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.382745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:50.382839 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.382818 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:50.383022 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.382856 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls podName:5958b543-bd6a-4291-b525-944cc570ea22 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:52.38284347 +0000 UTC m=+41.264568604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzggl" (UID: "5958b543-bd6a-4291-b525-944cc570ea22") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:50.484416 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.484272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:50.484612 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.484597 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:50.484685 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.484660 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls podName:b76ec3fb-81ff-4ef8-9854-d1a957f0020c nodeName:}" failed. No retries permitted until 2026-09-10 17:41:52.484645107 +0000 UTC m=+41.366370229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rbdfr" (UID: "b76ec3fb-81ff-4ef8-9854-d1a957f0020c") : secret "samples-operator-tls" not found Sep 10 17:41:50.513677 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.513648 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:50.585139 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.585117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:50.585252 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.585218 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:50.585252 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.585244 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcb66bc9-ql8cz: secret "image-registry-tls" not found Sep 10 17:41:50.585330 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:50.585290 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls podName:f26f85db-bbe5-4cc7-a49e-1eb8268f24e9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:52.58527642 +0000 UTC m=+41.467001553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls") pod "image-registry-7bcb66bc9-ql8cz" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9") : secret "image-registry-tls" not found Sep 10 17:41:50.613876 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.613849 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:50.820544 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.820505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" event={"ID":"b7d62301-70d4-4c47-b170-e7b5dcebae76","Type":"ContainerStarted","Data":"2f6902186fb6eaba74f31af10bbd89a9bceecdcbfaf50b97b76ca6897d89b2f1"} Sep 10 17:41:50.841107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.841058 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cc8xm" podStartSLOduration=1.546209918 podStartE2EDuration="2.841042861s" totalImagesPullingTime="1.294832943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:50.839700269 +0000 UTC m=+39.721425410" watchObservedRunningTime="2026-09-10 17:41:50.841042861 +0000 UTC m=+39.722768002" Sep 10 17:41:50.924793 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:50.924751 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:51.030974 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:51.030939 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:52.400261 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.400172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:52.400261 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.400240 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:52.400725 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.400336 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:52.400725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.400350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:52.400725 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.400406 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.400384371 +0000 UTC m=+45.282109509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : configmap references non-existent config key: service-ca.crt Sep 10 17:41:52.400725 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.400429 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls podName:5958b543-bd6a-4291-b525-944cc570ea22 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.40041974 +0000 UTC m=+45.282144866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzggl" (UID: "5958b543-bd6a-4291-b525-944cc570ea22") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:52.400725 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.400463 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 17:41:52.400725 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.400517 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.400503197 +0000 UTC m=+45.282228333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : secret "router-metrics-certs-default" not found Sep 10 17:41:52.501569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.501539 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:52.501711 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.501671 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:52.501752 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.501724 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls podName:b76ec3fb-81ff-4ef8-9854-d1a957f0020c nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.501710833 +0000 UTC m=+45.383435979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rbdfr" (UID: "b76ec3fb-81ff-4ef8-9854-d1a957f0020c") : secret "samples-operator-tls" not found Sep 10 17:41:52.602612 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.602582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:52.602754 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.602729 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:52.602754 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.602748 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcb66bc9-ql8cz: secret "image-registry-tls" not found Sep 10 17:41:52.602823 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.602794 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls podName:f26f85db-bbe5-4cc7-a49e-1eb8268f24e9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.602781276 +0000 UTC m=+45.484506408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls") pod "image-registry-7bcb66bc9-ql8cz" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9") : secret "image-registry-tls" not found Sep 10 17:41:52.703804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.703780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:41:52.703953 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.703935 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:52.704006 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:52.703996 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls podName:a1cde6b8-5c75-434a-8880-1db69cfd7ebe nodeName:}" failed. No retries permitted until 2026-09-10 17:42:00.703980001 +0000 UTC m=+49.585705126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls") pod "dns-default-6tl2g" (UID: "a1cde6b8-5c75-434a-8880-1db69cfd7ebe") : secret "dns-default-metrics-tls" not found Sep 10 17:41:52.830443 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.830411 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" event={"ID":"de119af3-a57d-41b9-a334-63c626cc1d27","Type":"ContainerStarted","Data":"fd3ef908f4dcf5fa4bb2309c0d0809a9c89f5455d1ca059105f9f5d311a3b12f"} Sep 10 17:41:52.832096 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.832072 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" event={"ID":"e8ef2482-361b-49bd-8f72-a594191b49fc","Type":"ContainerStarted","Data":"7b611dcc6bd055e2ed60d0e655e20968f9ed12fcc711dc7160365a4ed295ad68"} Sep 10 17:41:52.850143 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.850095 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" podStartSLOduration=1.811107916 podStartE2EDuration="4.850078345s" totalImagesPullingTime="3.038970429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:52.849043201 +0000 UTC m=+41.730768343" watchObservedRunningTime="2026-09-10 17:41:52.850078345 +0000 UTC m=+41.731803488" Sep 10 17:41:52.872038 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:52.871993 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" podStartSLOduration=2.128297806 podStartE2EDuration="4.871975441s" totalImagesPullingTime="2.743677635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:52.870300147 +0000 UTC m=+41.752025299" watchObservedRunningTime="2026-09-10 17:41:52.871975441 +0000 UTC m=+41.753700584" Sep 10 17:41:53.468296 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:53.468263 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:53.571147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:53.571104 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:53.955712 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:53.955682 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:54.047128 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.047103 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:54.153011 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.152967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:54.245132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.245055 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:54.797586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.797559 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ktl4g"] Sep 10 17:41:54.798067 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.798033 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:54.827512 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.827482 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktl4g"] Sep 10 17:41:54.827645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.827616 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:54.831574 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.831544 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 17:41:54.831937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.831918 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fk6pf\"" Sep 10 17:41:54.832036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.831914 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 17:41:54.833732 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.833717 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 17:41:54.834742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.834724 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 17:41:54.866640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.866614 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:54.922490 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.922456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae938dde-0a01-4dc8-a034-10b6b953f26a-data-volume\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:54.922614 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.922511 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thb7f\" (UniqueName: \"kubernetes.io/projected/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-api-access-thb7f\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:54.922614 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.922551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae938dde-0a01-4dc8-a034-10b6b953f26a-crio-socket\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:54.922614 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.922577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:54.922710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:54.922639 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.002510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.002482 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.023702 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.023679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae938dde-0a01-4dc8-a034-10b6b953f26a-data-volume\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.023791 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.023730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thb7f\" (UniqueName: \"kubernetes.io/projected/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-api-access-thb7f\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.023791 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.023774 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae938dde-0a01-4dc8-a034-10b6b953f26a-crio-socket\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.023873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.023803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.023873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.023852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.024006 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:55.023951 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:55.024006 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:55.023958 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 17:41:55.024087 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:55.024010 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls podName:ae938dde-0a01-4dc8-a034-10b6b953f26a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:55.523995012 +0000 UTC m=+44.405720147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktl4g" (UID: "ae938dde-0a01-4dc8-a034-10b6b953f26a") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:55.024087 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:55.024029 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-rbac-proxy-cm podName:ae938dde-0a01-4dc8-a034-10b6b953f26a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:55.524018803 +0000 UTC m=+44.405743921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktl4g" (UID: "ae938dde-0a01-4dc8-a034-10b6b953f26a") : configmap "kube-rbac-proxy" not found Sep 10 17:41:55.024087 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.024036 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae938dde-0a01-4dc8-a034-10b6b953f26a-data-volume\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.024087 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.024040 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae938dde-0a01-4dc8-a034-10b6b953f26a-crio-socket\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.038323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.038300 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thb7f\" (UniqueName: \"kubernetes.io/projected/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-api-access-thb7f\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.119970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.119895 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.298465 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.298433 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.354075 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.354042 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.501111 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.501076 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.528589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.528568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.528697 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.528630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.528737 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:55.528716 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:55.528787 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:55.528777 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls podName:ae938dde-0a01-4dc8-a034-10b6b953f26a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.528761556 +0000 UTC m=+45.410486675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktl4g" (UID: "ae938dde-0a01-4dc8-a034-10b6b953f26a") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:55.529097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.529081 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae938dde-0a01-4dc8-a034-10b6b953f26a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:55.799782 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.799667 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.854719 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:55.854686 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:56.436482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.436448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:56.436650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.436503 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:41:56.436650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.436539 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:41:56.436650 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.436612 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 17:41:56.436650 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.436643 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:42:04.436624024 +0000 UTC m=+53.318349166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : configmap references non-existent config key: service-ca.crt Sep 10 17:41:56.436830 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.436660 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:56.436830 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.436673 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:42:04.436654491 +0000 UTC m=+53.318379634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : secret "router-metrics-certs-default" not found Sep 10 17:41:56.436830 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.436698 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls podName:5958b543-bd6a-4291-b525-944cc570ea22 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:04.436687748 +0000 UTC m=+53.318412886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzggl" (UID: "5958b543-bd6a-4291-b525-944cc570ea22") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:56.535411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.535385 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6b7qr_79157419-0d17-4cb0-8ebe-e1efbb0fd39c/dns-node-resolver/0.log" Sep 10 17:41:56.537717 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.537697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:41:56.537778 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.537725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:56.537822 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.537812 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:56.537859 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.537855 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls podName:ae938dde-0a01-4dc8-a034-10b6b953f26a nodeName:}" failed. No retries permitted until 2026-09-10 17:41:58.537844099 +0000 UTC m=+47.419569218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktl4g" (UID: "ae938dde-0a01-4dc8-a034-10b6b953f26a") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:56.537900 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.537812 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:56.537900 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.537884 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls podName:b76ec3fb-81ff-4ef8-9854-d1a957f0020c nodeName:}" failed. No retries permitted until 2026-09-10 17:42:04.53787812 +0000 UTC m=+53.419603238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rbdfr" (UID: "b76ec3fb-81ff-4ef8-9854-d1a957f0020c") : secret "samples-operator-tls" not found Sep 10 17:41:56.638399 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.638372 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:41:56.638510 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.638494 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:56.638556 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.638512 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcb66bc9-ql8cz: secret "image-registry-tls" not found Sep 10 17:41:56.638556 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:56.638554 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls podName:f26f85db-bbe5-4cc7-a49e-1eb8268f24e9 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:04.638542402 +0000 UTC m=+53.520267538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls") pod "image-registry-7bcb66bc9-ql8cz" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9") : secret "image-registry-tls" not found Sep 10 17:41:56.726208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:56.726178 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:57.934859 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:57.934834 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mh7pm_067eb756-a003-43b7-8693-116193f3ae2c/node-ca/0.log" Sep 10 17:41:58.335248 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:58.335160 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5tvgn_94aed250-dd5b-4c9c-aeb5-9178dde79af0/serve-healthcheck-canary/0.log" Sep 10 17:41:58.553758 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:58.553719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:41:58.553936 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:58.553868 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:58.553936 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:41:58.553933 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls podName:ae938dde-0a01-4dc8-a034-10b6b953f26a nodeName:}" failed. No retries permitted until 2026-09-10 17:42:02.553918322 +0000 UTC m=+51.435643457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktl4g" (UID: "ae938dde-0a01-4dc8-a034-10b6b953f26a") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:58.975997 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:58.975967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.139995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.139968 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.240059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.239990 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.341014 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.340992 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.466398 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.466373 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.628112 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.628043 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.731260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.731220 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.839122 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.839097 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:59.889510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:41:59.889459 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:00.014615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:00.014594 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:00.405623 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:00.405594 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:00.771356 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:00.771330 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:42:00.771515 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:00.771486 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:42:00.771560 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:00.771540 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls podName:a1cde6b8-5c75-434a-8880-1db69cfd7ebe nodeName:}" failed. No retries permitted until 2026-09-10 17:42:16.771524648 +0000 UTC m=+65.653249777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls") pod "dns-default-6tl2g" (UID: "a1cde6b8-5c75-434a-8880-1db69cfd7ebe") : secret "dns-default-metrics-tls" not found Sep 10 17:42:01.007482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:01.007447 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:01.813953 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:01.813919 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:02.210858 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:02.210821 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:02.412696 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:02.412668 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:02.591277 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:02.591202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:42:02.591402 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:02.591345 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:42:02.591441 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:02.591408 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls podName:ae938dde-0a01-4dc8-a034-10b6b953f26a nodeName:}" failed. No retries permitted until 2026-09-10 17:42:10.591387471 +0000 UTC m=+59.473112591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktl4g" (UID: "ae938dde-0a01-4dc8-a034-10b6b953f26a") : secret "insights-runtime-extractor-tls" not found Sep 10 17:42:02.613024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:02.612992 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.004977 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.004945 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.060018 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.059988 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.395333 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.395263 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.449107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.449065 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.510249 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.510197 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.810777 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.810750 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:03.861391 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:03.861370 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:04.409178 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.409148 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:04.505656 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.505627 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:04.505785 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.505687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:42:04.505785 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.505714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:04.505861 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:04.505837 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:42:04.505861 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:04.505850 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle podName:511026f7-b2e0-47ae-8d79-51eab0beb87a nodeName:}" failed. No retries permitted until 2026-09-10 17:42:20.505837264 +0000 UTC m=+69.387562396 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle") pod "router-default-765484f768-8d5m8" (UID: "511026f7-b2e0-47ae-8d79-51eab0beb87a") : configmap references non-existent config key: service-ca.crt Sep 10 17:42:04.505942 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:04.505888 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls podName:5958b543-bd6a-4291-b525-944cc570ea22 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:20.505875241 +0000 UTC m=+69.387600359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzggl" (UID: "5958b543-bd6a-4291-b525-944cc570ea22") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:42:04.508043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.508020 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511026f7-b2e0-47ae-8d79-51eab0beb87a-metrics-certs\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:04.606470 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.606443 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:42:04.607283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.607258 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:04.608775 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.608755 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76ec3fb-81ff-4ef8-9854-d1a957f0020c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rbdfr\" (UID: \"b76ec3fb-81ff-4ef8-9854-d1a957f0020c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:42:04.707568 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.707517 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:04.711688 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.711664 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"image-registry-7bcb66bc9-ql8cz\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:04.807433 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.807404 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:04.892580 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.892558 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" Sep 10 17:42:04.994823 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:04.994760 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:05.158431 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.158407 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr"] Sep 10 17:42:05.265161 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.265102 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bcb66bc9-ql8cz"] Sep 10 17:42:05.267364 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:05.267339 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26f85db_bbe5_4cc7_a49e_1eb8268f24e9.slice/crio-4b86c2f0abeb1e03fbe9d57c37f214c31108f5e5a7d6cebf5bc42a51e1a92e4b WatchSource:0}: Error finding container 4b86c2f0abeb1e03fbe9d57c37f214c31108f5e5a7d6cebf5bc42a51e1a92e4b: Status 404 returned error can't find the container with id 4b86c2f0abeb1e03fbe9d57c37f214c31108f5e5a7d6cebf5bc42a51e1a92e4b Sep 10 17:42:05.625303 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.625214 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.811430 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.811400 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.864064 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.864025 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" event={"ID":"b76ec3fb-81ff-4ef8-9854-d1a957f0020c","Type":"ContainerStarted","Data":"a7e806c8e49759ef5087df255009e988fe431d52d316752b6153dcfc07d0052b"} Sep 10 17:42:05.865948 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.865919 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" event={"ID":"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9","Type":"ContainerStarted","Data":"1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5"} Sep 10 17:42:05.866085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.865968 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" event={"ID":"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9","Type":"ContainerStarted","Data":"4b86c2f0abeb1e03fbe9d57c37f214c31108f5e5a7d6cebf5bc42a51e1a92e4b"} Sep 10 17:42:05.866085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.866067 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:05.887760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:05.887689 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" podStartSLOduration=17.887676256 podStartE2EDuration="17.887676256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:05.887495725 +0000 UTC m=+54.769220868" watchObservedRunningTime="2026-09-10 17:42:05.887676256 +0000 UTC m=+54.769401388" Sep 10 17:42:06.212748 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:06.212717 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:06.409455 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:06.409423 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:06.616072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:06.615994 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.007137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:07.007115 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.606719 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:07.606689 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.820980 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:07.820951 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.873558 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:07.873489 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" event={"ID":"b76ec3fb-81ff-4ef8-9854-d1a957f0020c","Type":"ContainerStarted","Data":"b1f1d65c7696fc8f25c3c75a6f7f04b0182b1c5f9141537c495264db6be56524"} Sep 10 17:42:07.873678 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:07.873564 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" event={"ID":"b76ec3fb-81ff-4ef8-9854-d1a957f0020c","Type":"ContainerStarted","Data":"972d7571a4de6098a2d993a690b12edcbdbfdbf29d7ffb3e1fb1a9369e09f5f3"} Sep 10 17:42:07.893913 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:07.893866 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rbdfr" podStartSLOduration=18.110511553 podStartE2EDuration="19.89385139s" totalImagesPullingTime="1.783339837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:07.892120489 +0000 UTC m=+56.773845627" watchObservedRunningTime="2026-09-10 17:42:07.89385139 +0000 UTC m=+56.775576530" Sep 10 17:42:08.012417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:08.012391 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.222916 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:09.222883 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.414194 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:09.414168 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.613108 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:09.613026 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.809893 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:09.809861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.011791 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.011767 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.536126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.536098 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.637105 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.637072 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.656195 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.656168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:42:10.658352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.658331 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae938dde-0a01-4dc8-a034-10b6b953f26a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktl4g\" (UID: \"ae938dde-0a01-4dc8-a034-10b6b953f26a\") " pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:42:10.736500 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.736478 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktl4g" Sep 10 17:42:10.736589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.736520 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.778302 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:10.778276 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-22nhh" Sep 10 17:42:11.016526 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.016500 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktl4g"] Sep 10 17:42:11.019370 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:11.019344 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae938dde_0a01_4dc8_a034_10b6b953f26a.slice/crio-5ff507ca65eb25d4a7bac95208f14bb3815c6d5f013db2d5ed4236e84df03b9c WatchSource:0}: Error finding container 5ff507ca65eb25d4a7bac95208f14bb3815c6d5f013db2d5ed4236e84df03b9c: Status 404 returned error can't find the container with id 5ff507ca65eb25d4a7bac95208f14bb3815c6d5f013db2d5ed4236e84df03b9c Sep 10 17:42:11.029287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.029257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:11.206379 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.206348 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:11.307446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.307382 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:11.888955 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.888925 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktl4g" event={"ID":"ae938dde-0a01-4dc8-a034-10b6b953f26a","Type":"ContainerStarted","Data":"c8f501dfdfe63b2faca43d4d8332da162e51646af961901a5b2f1f5e2283d6b6"} Sep 10 17:42:11.889287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.888978 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktl4g" event={"ID":"ae938dde-0a01-4dc8-a034-10b6b953f26a","Type":"ContainerStarted","Data":"452ababa5383dee765f0aa85bda1f62c605ebf4fe18b09c0ff109626b1fd2082"} Sep 10 17:42:11.889287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:11.889000 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktl4g" event={"ID":"ae938dde-0a01-4dc8-a034-10b6b953f26a","Type":"ContainerStarted","Data":"5ff507ca65eb25d4a7bac95208f14bb3815c6d5f013db2d5ed4236e84df03b9c"} Sep 10 17:42:12.046012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:12.045986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:12.105361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:12.105334 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:12.219487 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:12.219431 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:12.418861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:12.418752 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:12.617927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:12.617882 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:12.718352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:12.718317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:13.896676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:13.896638 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktl4g" event={"ID":"ae938dde-0a01-4dc8-a034-10b6b953f26a","Type":"ContainerStarted","Data":"4bb80dd88840136766039bb670a238d98e8bdcdd40b847836c1f5d4c42e5fa50"} Sep 10 17:42:13.915542 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:13.915498 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ktl4g" podStartSLOduration=18.141995783 podStartE2EDuration="19.915484174s" totalImagesPullingTime="1.773488391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:13.914957524 +0000 UTC m=+62.796682668" watchObservedRunningTime="2026-09-10 17:42:13.915484174 +0000 UTC m=+62.797209315" Sep 10 17:42:14.144691 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:14.144663 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:14.200213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:14.200186 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:16.806514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:16.806484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:42:16.808819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:16.808784 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1cde6b8-5c75-434a-8880-1db69cfd7ebe-metrics-tls\") pod \"dns-default-6tl2g\" (UID: \"a1cde6b8-5c75-434a-8880-1db69cfd7ebe\") " pod="openshift-dns/dns-default-6tl2g" Sep 10 17:42:17.068568 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.068490 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d7j7h\"" Sep 10 17:42:17.075834 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.075815 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6tl2g" Sep 10 17:42:17.310716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.310683 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:42:17.313437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.313420 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 17:42:17.323397 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.323354 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd31ec3-6041-40ea-a2cd-a991b38024c7-original-pull-secret\") pod \"global-pull-secret-syncer-vqz5s\" (UID: \"2cd31ec3-6041-40ea-a2cd-a991b38024c7\") " pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:42:17.377451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.377427 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6tl2g"] Sep 10 17:42:17.380668 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:17.380645 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cde6b8_5c75_434a_8880_1db69cfd7ebe.slice/crio-713a9ad6a80dc852f06eb57930455b0bcca3b80f5addd7fe6cd50677d0a63971 WatchSource:0}: Error finding container 713a9ad6a80dc852f06eb57930455b0bcca3b80f5addd7fe6cd50677d0a63971: Status 404 returned error can't find the container with id 713a9ad6a80dc852f06eb57930455b0bcca3b80f5addd7fe6cd50677d0a63971 Sep 10 17:42:17.411921 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.411899 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:42:17.418418 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.418400 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 17:42:17.424142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.424126 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9588d65-3514-42ee-9252-56457aeeac79-metrics-certs\") pod \"network-metrics-daemon-g2jbw\" (UID: \"d9588d65-3514-42ee-9252-56457aeeac79\") " pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:42:17.430488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.430469 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hkln9\"" Sep 10 17:42:17.437805 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.437784 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2jbw" Sep 10 17:42:17.437884 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.437866 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vqz5s" Sep 10 17:42:17.519244 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.517608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:42:17.530265 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.528560 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 17:42:17.541372 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.541349 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 17:42:17.550732 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.550706 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k954\" (UniqueName: \"kubernetes.io/projected/ac66f28e-0601-4dcc-99d7-2a298bc5d51f-kube-api-access-6k954\") pod \"network-check-target-prwhp\" (UID: \"ac66f28e-0601-4dcc-99d7-2a298bc5d51f\") " pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:42:17.557357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.557337 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bcb66bc9-ql8cz"] Sep 10 17:42:17.616805 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.616744 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-vdf6l"] Sep 10 17:42:17.619316 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.619294 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.622119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.622096 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 17:42:17.622380 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.622359 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 17:42:17.622470 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.622408 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vkvjk\"" Sep 10 17:42:17.634135 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.634113 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-vdf6l"] Sep 10 17:42:17.646404 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.646373 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:17.718533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.718508 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e06c3493-58ed-4a5a-b1f0-a9bd8d673ead-nginx-conf\") pod \"networking-console-plugin-79977f9844-vdf6l\" (UID: \"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.718628 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.718580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e06c3493-58ed-4a5a-b1f0-a9bd8d673ead-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vdf6l\" (UID: \"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.722638 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.722600 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2jbw"] Sep 10 17:42:17.723486 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.723442 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vqz5s"] Sep 10 17:42:17.725265 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:17.725246 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9588d65_3514_42ee_9252_56457aeeac79.slice/crio-20c4bda7863ddb7a6604529eb1129fdb4bd82febc31d9cfa5d0162db6e04a53e WatchSource:0}: Error finding container 20c4bda7863ddb7a6604529eb1129fdb4bd82febc31d9cfa5d0162db6e04a53e: Status 404 returned error can't find the container with id 20c4bda7863ddb7a6604529eb1129fdb4bd82febc31d9cfa5d0162db6e04a53e Sep 10 17:42:17.725879 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:17.725816 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd31ec3_6041_40ea_a2cd_a991b38024c7.slice/crio-ff093f527ec86598242a776d19b67d278b9058836ae874ccfa2624a7ff0c1a87 WatchSource:0}: Error finding container ff093f527ec86598242a776d19b67d278b9058836ae874ccfa2624a7ff0c1a87: Status 404 returned error can't find the container with id ff093f527ec86598242a776d19b67d278b9058836ae874ccfa2624a7ff0c1a87 Sep 10 17:42:17.727788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.727764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:17.736281 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.736255 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fbgqn\"" Sep 10 17:42:17.744580 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.744559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:42:17.819682 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.819651 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e06c3493-58ed-4a5a-b1f0-a9bd8d673ead-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vdf6l\" (UID: \"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.820060 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.819712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e06c3493-58ed-4a5a-b1f0-a9bd8d673ead-nginx-conf\") pod \"networking-console-plugin-79977f9844-vdf6l\" (UID: \"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.820602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.820579 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e06c3493-58ed-4a5a-b1f0-a9bd8d673ead-nginx-conf\") pod \"networking-console-plugin-79977f9844-vdf6l\" (UID: \"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.822537 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.822513 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e06c3493-58ed-4a5a-b1f0-a9bd8d673ead-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-vdf6l\" (UID: \"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead\") " pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:17.826924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.826866 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:17.910864 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.910787 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2jbw" event={"ID":"d9588d65-3514-42ee-9252-56457aeeac79","Type":"ContainerStarted","Data":"20c4bda7863ddb7a6604529eb1129fdb4bd82febc31d9cfa5d0162db6e04a53e"} Sep 10 17:42:17.912129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.912105 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6tl2g" event={"ID":"a1cde6b8-5c75-434a-8880-1db69cfd7ebe","Type":"ContainerStarted","Data":"713a9ad6a80dc852f06eb57930455b0bcca3b80f5addd7fe6cd50677d0a63971"} Sep 10 17:42:17.913465 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.913440 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vqz5s" event={"ID":"2cd31ec3-6041-40ea-a2cd-a991b38024c7","Type":"ContainerStarted","Data":"ff093f527ec86598242a776d19b67d278b9058836ae874ccfa2624a7ff0c1a87"} Sep 10 17:42:17.921683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.921655 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:17.928483 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:17.928462 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" Sep 10 17:42:18.022796 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:18.022767 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-prwhp"] Sep 10 17:42:18.025671 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:18.025645 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac66f28e_0601_4dcc_99d7_2a298bc5d51f.slice/crio-1e7b3d4cb7ba024a523b073997734968edc422a0399324971c4cdcb88c6879b1 WatchSource:0}: Error finding container 1e7b3d4cb7ba024a523b073997734968edc422a0399324971c4cdcb88c6879b1: Status 404 returned error can't find the container with id 1e7b3d4cb7ba024a523b073997734968edc422a0399324971c4cdcb88c6879b1 Sep 10 17:42:18.215264 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:18.215217 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-vdf6l"] Sep 10 17:42:18.219828 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:18.219800 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06c3493_58ed_4a5a_b1f0_a9bd8d673ead.slice/crio-8deea929a3154503e9fd8d12af0afd4abe9dc4fb9bd8c6a9e5a38b0f9fc1194a WatchSource:0}: Error finding container 8deea929a3154503e9fd8d12af0afd4abe9dc4fb9bd8c6a9e5a38b0f9fc1194a: Status 404 returned error can't find the container with id 8deea929a3154503e9fd8d12af0afd4abe9dc4fb9bd8c6a9e5a38b0f9fc1194a Sep 10 17:42:18.442113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:18.442080 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:18.496524 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:18.496438 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:18.919523 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:18.919427 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" event={"ID":"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead","Type":"ContainerStarted","Data":"8deea929a3154503e9fd8d12af0afd4abe9dc4fb9bd8c6a9e5a38b0f9fc1194a"} Sep 10 17:42:18.921265 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:18.921211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-prwhp" event={"ID":"ac66f28e-0601-4dcc-99d7-2a298bc5d51f","Type":"ContainerStarted","Data":"1e7b3d4cb7ba024a523b073997734968edc422a0399324971c4cdcb88c6879b1"} Sep 10 17:42:19.503500 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.503435 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:19.555943 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.555861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:19.928566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.927725 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2jbw" event={"ID":"d9588d65-3514-42ee-9252-56457aeeac79","Type":"ContainerStarted","Data":"4a02e6dda2a73ba875acf21624e674fb4529bc7f85483d2cc093db75659771ec"} Sep 10 17:42:19.928566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.927790 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2jbw" event={"ID":"d9588d65-3514-42ee-9252-56457aeeac79","Type":"ContainerStarted","Data":"208fb80558ec8bb4560de6e8ba931de487f710ec4e05c462680742e68aefa805"} Sep 10 17:42:19.931192 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.930732 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6tl2g" event={"ID":"a1cde6b8-5c75-434a-8880-1db69cfd7ebe","Type":"ContainerStarted","Data":"1f9f99f3568c616a7a3fa7b63f38b80b3e82c76e3ec2fedddadedf4117526cb0"} Sep 10 17:42:19.931192 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.930785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6tl2g" event={"ID":"a1cde6b8-5c75-434a-8880-1db69cfd7ebe","Type":"ContainerStarted","Data":"112576f30a167ac2452971867c9568171e29d5418423c21e158a6b3e1992bed4"} Sep 10 17:42:19.931192 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.931169 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6tl2g" Sep 10 17:42:19.945842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.945789 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g2jbw" podStartSLOduration=67.368953805 podStartE2EDuration="1m8.945771882s" totalImagesPullingTime="1.576818077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:19.945145925 +0000 UTC m=+68.826871067" watchObservedRunningTime="2026-09-10 17:42:19.945771882 +0000 UTC m=+68.827497024" Sep 10 17:42:19.964833 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:19.964779 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6tl2g" podStartSLOduration=34.035945028 podStartE2EDuration="35.964761332s" totalImagesPullingTime="1.928816304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:19.964619093 +0000 UTC m=+68.846344234" watchObservedRunningTime="2026-09-10 17:42:19.964761332 +0000 UTC m=+68.846486474" Sep 10 17:42:20.045072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.045013 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:20.103795 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.103742 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:20.542735 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.542649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:42:20.542735 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.542712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:20.543495 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.543466 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/511026f7-b2e0-47ae-8d79-51eab0beb87a-service-ca-bundle\") pod \"router-default-765484f768-8d5m8\" (UID: \"511026f7-b2e0-47ae-8d79-51eab0beb87a\") " pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:20.545753 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.545728 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5958b543-bd6a-4291-b525-944cc570ea22-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzggl\" (UID: \"5958b543-bd6a-4291-b525-944cc570ea22\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:42:20.713147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.713114 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g8lbh\"" Sep 10 17:42:20.720744 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.720720 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:20.731897 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.731868 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8k98b\"" Sep 10 17:42:20.738167 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.738135 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" Sep 10 17:42:20.940193 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.940154 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" event={"ID":"e06c3493-58ed-4a5a-b1f0-a9bd8d673ead","Type":"ContainerStarted","Data":"0bd1948d3ebd692ea4be886b545e7427b487241a0c3a188e33d839cbd2c4c4f6"} Sep 10 17:42:20.964524 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:20.964473 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-vdf6l" podStartSLOduration=1.9755006750000002 podStartE2EDuration="3.964453243s" totalImagesPullingTime="1.988952568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:20.963591903 +0000 UTC m=+69.845317043" watchObservedRunningTime="2026-09-10 17:42:20.964453243 +0000 UTC m=+69.846178384" Sep 10 17:42:21.064748 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:21.064718 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.162153 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:21.162119 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.263548 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:21.263441 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.243949 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.243911 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.300838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.300782 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.952014 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.951973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-prwhp" event={"ID":"ac66f28e-0601-4dcc-99d7-2a298bc5d51f","Type":"ContainerStarted","Data":"f188113ae30926ea64a7db619ef4c0375ec12e21d5a2d8d258e7a23f1f6625d8"} Sep 10 17:42:22.952218 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.952088 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:42:22.953648 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.953620 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vqz5s" event={"ID":"2cd31ec3-6041-40ea-a2cd-a991b38024c7","Type":"ContainerStarted","Data":"7195338c11269b2b57c6c52d83b5c96429322900ec6f3c3cfb21b6d28c91f1ba"} Sep 10 17:42:22.970622 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.970567 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-prwhp" podStartSLOduration=67.239231634 podStartE2EDuration="1m11.970554901s" totalImagesPullingTime="4.731323267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:22.969624755 +0000 UTC m=+71.851349898" watchObservedRunningTime="2026-09-10 17:42:22.970554901 +0000 UTC m=+71.852280043" Sep 10 17:42:22.986067 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:22.986025 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vqz5s" podStartSLOduration=66.947624626 podStartE2EDuration="1m11.98601249s" totalImagesPullingTime="5.038387864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:22.983841894 +0000 UTC m=+71.865567035" watchObservedRunningTime="2026-09-10 17:42:22.98601249 +0000 UTC m=+71.867737630" Sep 10 17:42:23.023887 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:23.023817 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl"] Sep 10 17:42:23.026331 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:23.026311 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-765484f768-8d5m8"] Sep 10 17:42:23.026744 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:23.026725 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5958b543_bd6a_4291_b525_944cc570ea22.slice/crio-723a6ab5d53b8696f489ea6b8a7ff17976a748b43d770c931c5bffff541ce550 WatchSource:0}: Error finding container 723a6ab5d53b8696f489ea6b8a7ff17976a748b43d770c931c5bffff541ce550: Status 404 returned error can't find the container with id 723a6ab5d53b8696f489ea6b8a7ff17976a748b43d770c931c5bffff541ce550 Sep 10 17:42:23.029458 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:23.029429 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511026f7_b2e0_47ae_8d79_51eab0beb87a.slice/crio-84d318df7910f380cbc9ff54da65483559e4b7ed2e8e45493f1d16795b88633b WatchSource:0}: Error finding container 84d318df7910f380cbc9ff54da65483559e4b7ed2e8e45493f1d16795b88633b: Status 404 returned error can't find the container with id 84d318df7910f380cbc9ff54da65483559e4b7ed2e8e45493f1d16795b88633b Sep 10 17:42:23.960072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:23.960002 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" event={"ID":"5958b543-bd6a-4291-b525-944cc570ea22","Type":"ContainerStarted","Data":"723a6ab5d53b8696f489ea6b8a7ff17976a748b43d770c931c5bffff541ce550"} Sep 10 17:42:23.961829 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:23.961784 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-765484f768-8d5m8" event={"ID":"511026f7-b2e0-47ae-8d79-51eab0beb87a","Type":"ContainerStarted","Data":"0c45bfbd970ad8a49a80c33e7c441de73d2206e5e81951ddb7202d55d97982ec"} Sep 10 17:42:23.961967 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:23.961839 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-765484f768-8d5m8" event={"ID":"511026f7-b2e0-47ae-8d79-51eab0beb87a","Type":"ContainerStarted","Data":"84d318df7910f380cbc9ff54da65483559e4b7ed2e8e45493f1d16795b88633b"} Sep 10 17:42:23.983137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:23.983093 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-765484f768-8d5m8" podStartSLOduration=35.98307873 podStartE2EDuration="35.98307873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:23.983070178 +0000 UTC m=+72.864795321" watchObservedRunningTime="2026-09-10 17:42:23.98307873 +0000 UTC m=+72.864803870" Sep 10 17:42:24.720941 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:24.720901 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:24.723676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:24.723647 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:24.965874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:24.965847 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:24.966999 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:24.966980 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-765484f768-8d5m8" Sep 10 17:42:25.456511 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:25.456479 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:25.565330 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:25.565297 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:25.631508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:25.631481 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:25.687837 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:25.687811 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:25.970236 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:25.970202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" event={"ID":"5958b543-bd6a-4291-b525-944cc570ea22","Type":"ContainerStarted","Data":"11aa45aefb027f45b0a79b505e4d365a5275e2b21b4d4cea90f3e373f16648b5"} Sep 10 17:42:25.988512 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:25.988464 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzggl" podStartSLOduration=35.885137772 podStartE2EDuration="37.988450632s" totalImagesPullingTime="2.10331286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:25.987319932 +0000 UTC m=+74.869045074" watchObservedRunningTime="2026-09-10 17:42:25.988450632 +0000 UTC m=+74.870175802" Sep 10 17:42:26.018539 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:26.018516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:27.563758 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:27.563733 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:27.637288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:27.637253 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:27.693023 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:27.692994 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:28.510752 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:28.510719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:28.566567 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:28.566536 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:29.137601 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:29.137567 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:29.238734 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:29.238702 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:30.027858 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.027827 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8474f697f7-twrsz"] Sep 10 17:42:30.030249 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.030215 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.033465 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.033434 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 17:42:30.034422 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.034407 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 17:42:30.034488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.034446 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 17:42:30.034694 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.034670 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 17:42:30.034787 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.034708 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 17:42:30.034787 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.034743 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fz4lp\"" Sep 10 17:42:30.034978 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.034951 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 17:42:30.035053 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.035037 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 17:42:30.040622 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.040601 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8474f697f7-twrsz"] Sep 10 17:42:30.118050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.118025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-service-ca\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.118148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.118057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-serving-cert\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.118148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.118077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-oauth-config\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.118148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.118096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-config\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.118313 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.118157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-oauth-serving-cert\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.118313 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.118176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87ts8\" (UniqueName: \"kubernetes.io/projected/72611a31-79ad-40dd-a69b-e83b1f12a30b-kube-api-access-87ts8\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.219342 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.219316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-service-ca\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.219437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.219350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-serving-cert\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.219437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.219370 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-oauth-config\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.219437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.219385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-config\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.219437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.219414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-oauth-serving-cert\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.219581 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.219439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ts8\" (UniqueName: \"kubernetes.io/projected/72611a31-79ad-40dd-a69b-e83b1f12a30b-kube-api-access-87ts8\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.220164 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.220132 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-oauth-serving-cert\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.220285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.220141 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-config\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.220285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.220141 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-service-ca\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.221851 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.221826 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-oauth-config\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.222061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.222037 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-serving-cert\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.229012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.228989 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87ts8\" (UniqueName: \"kubernetes.io/projected/72611a31-79ad-40dd-a69b-e83b1f12a30b-kube-api-access-87ts8\") pod \"console-8474f697f7-twrsz\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.339137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.339077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:30.362124 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.362087 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:30.462410 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.462355 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:30.607166 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.607029 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8474f697f7-twrsz"] Sep 10 17:42:30.609602 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:30.609567 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72611a31_79ad_40dd_a69b_e83b1f12a30b.slice/crio-8d854aa339263e9bc68b6fb5d4597d4c388ab10ada6710d421d4b4958c1447ff WatchSource:0}: Error finding container 8d854aa339263e9bc68b6fb5d4597d4c388ab10ada6710d421d4b4958c1447ff: Status 404 returned error can't find the container with id 8d854aa339263e9bc68b6fb5d4597d4c388ab10ada6710d421d4b4958c1447ff Sep 10 17:42:30.945215 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.945189 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6tl2g" Sep 10 17:42:30.993002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:30.992976 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474f697f7-twrsz" event={"ID":"72611a31-79ad-40dd-a69b-e83b1f12a30b","Type":"ContainerStarted","Data":"8d854aa339263e9bc68b6fb5d4597d4c388ab10ada6710d421d4b4958c1447ff"} Sep 10 17:42:31.041051 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:31.041023 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:31.094854 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:31.094830 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:33.058332 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:33.058290 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:33.121748 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:33.121707 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:33.844656 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:33.844523 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:33.945983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:33.945956 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:34.005453 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.005425 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474f697f7-twrsz" event={"ID":"72611a31-79ad-40dd-a69b-e83b1f12a30b","Type":"ContainerStarted","Data":"71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017"} Sep 10 17:42:34.030101 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.030058 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8474f697f7-twrsz" podStartSLOduration=1.160826267 podStartE2EDuration="4.030044712s" totalImagesPullingTime="2.869218445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:34.029010714 +0000 UTC m=+82.910735856" watchObservedRunningTime="2026-09-10 17:42:34.030044712 +0000 UTC m=+82.911769852" Sep 10 17:42:34.048654 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.048632 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5755485f76-n4wrz"] Sep 10 17:42:34.050493 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.050475 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.059277 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.059259 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 17:42:34.064353 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.064332 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5755485f76-n4wrz"] Sep 10 17:42:34.151732 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.151672 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-trusted-ca-bundle\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.151732 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.151702 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-oauth-config\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.151732 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.151722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxvtd\" (UniqueName: \"kubernetes.io/projected/7835c975-e597-4338-b1e5-8cc5a1d3b279-kube-api-access-xxvtd\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.151919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.151771 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-serving-cert\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.151919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.151868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-oauth-serving-cert\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.152051 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.151995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-config\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.152051 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.152030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-service-ca\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252468 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252446 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-oauth-serving-cert\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-config\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252503 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-service-ca\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-trusted-ca-bundle\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-oauth-config\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxvtd\" (UniqueName: \"kubernetes.io/projected/7835c975-e597-4338-b1e5-8cc5a1d3b279-kube-api-access-xxvtd\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.252919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.252600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-serving-cert\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.253279 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.253252 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-oauth-serving-cert\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.253390 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.253284 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-config\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.253390 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.253252 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-service-ca\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.253529 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.253509 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-trusted-ca-bundle\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.255506 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.255483 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-oauth-config\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.255575 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.255555 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-serving-cert\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.261861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.261841 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxvtd\" (UniqueName: \"kubernetes.io/projected/7835c975-e597-4338-b1e5-8cc5a1d3b279-kube-api-access-xxvtd\") pod \"console-5755485f76-n4wrz\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.360912 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.360887 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:34.635998 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.635969 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5755485f76-n4wrz"] Sep 10 17:42:34.638659 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:34.638628 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7835c975_e597_4338_b1e5_8cc5a1d3b279.slice/crio-81bed16070a83fbdccb0f4af9c79decde506a93a7b1501bf2497d5c4f6ccf4f9 WatchSource:0}: Error finding container 81bed16070a83fbdccb0f4af9c79decde506a93a7b1501bf2497d5c4f6ccf4f9: Status 404 returned error can't find the container with id 81bed16070a83fbdccb0f4af9c79decde506a93a7b1501bf2497d5c4f6ccf4f9 Sep 10 17:42:34.655090 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.655037 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:34.753799 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:34.753769 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:35.010428 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:35.010394 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5755485f76-n4wrz" event={"ID":"7835c975-e597-4338-b1e5-8cc5a1d3b279","Type":"ContainerStarted","Data":"f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd"} Sep 10 17:42:35.010580 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:35.010446 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5755485f76-n4wrz" event={"ID":"7835c975-e597-4338-b1e5-8cc5a1d3b279","Type":"ContainerStarted","Data":"81bed16070a83fbdccb0f4af9c79decde506a93a7b1501bf2497d5c4f6ccf4f9"} Sep 10 17:42:35.035344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:35.035299 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5755485f76-n4wrz" podStartSLOduration=1.035284722 podStartE2EDuration="1.035284722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:35.034697485 +0000 UTC m=+83.916422626" watchObservedRunningTime="2026-09-10 17:42:35.035284722 +0000 UTC m=+83.917009862" Sep 10 17:42:35.283870 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:35.283799 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:35.383702 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:35.383670 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:37.110712 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:37.110669 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:37.518630 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:37.518596 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:37.621036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:37.621001 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:38.653088 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:38.653051 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:38.721507 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:38.721473 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:38.930244 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:38.930195 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.435433 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:39.435403 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.500554 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:39.500524 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.557976 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:39.557949 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.658403 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:39.658372 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:40.340155 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:40.340121 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:40.340408 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:40.340178 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:40.344882 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:40.344858 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:41.036283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:41.036257 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:42:41.191677 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:41.191644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:41.290662 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:41.290589 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:41.554197 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:41.554094 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:42.576770 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.576728 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" podUID="f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" containerName="registry" containerID="cri-o://1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5" gracePeriod=30 Sep 10 17:42:42.786996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.786962 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:42.826805 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.826753 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:42.921078 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921047 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-image-registry-private-configuration\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921082 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-installation-pull-secrets\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921125 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921181 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkb8c\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-kube-api-access-jkb8c\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921211 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-ca-trust-extracted\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921257 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-bound-sa-token\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921300 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-certificates\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921352 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-trusted-ca\") pod \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\" (UID: \"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9\") " Sep 10 17:42:42.921928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921839 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:42:42.921928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.921901 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:42:42.923590 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.923556 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:42:42.923702 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.923641 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:42:42.923783 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.923765 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:42:42.923842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.923821 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-kube-api-access-jkb8c" (OuterVolumeSpecName: "kube-api-access-jkb8c") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "kube-api-access-jkb8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:42:42.923884 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.923847 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:42:42.929778 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:42.929754 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" (UID: "f26f85db-bbe5-4cc7-a49e-1eb8268f24e9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:42:43.016327 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.016301 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.022585 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022563 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkb8c\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-kube-api-access-jkb8c\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022585 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022582 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-ca-trust-extracted\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022593 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-bound-sa-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022603 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-certificates\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022613 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-trusted-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022622 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-image-registry-private-configuration\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022631 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-installation-pull-secrets\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.022705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.022639 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9-registry-tls\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:42:43.040510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.040489 2565 generic.go:356] "Generic (PLEG): container finished" podID="f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" containerID="1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5" exitCode=0 Sep 10 17:42:43.040610 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.040541 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" Sep 10 17:42:43.040674 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.040542 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" event={"ID":"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9","Type":"ContainerDied","Data":"1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5"} Sep 10 17:42:43.040720 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.040690 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcb66bc9-ql8cz" event={"ID":"f26f85db-bbe5-4cc7-a49e-1eb8268f24e9","Type":"ContainerDied","Data":"4b86c2f0abeb1e03fbe9d57c37f214c31108f5e5a7d6cebf5bc42a51e1a92e4b"} Sep 10 17:42:43.041637 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.041619 2565 scope.go:117] "RemoveContainer" containerID="1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5" Sep 10 17:42:43.049858 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.049843 2565 scope.go:117] "RemoveContainer" containerID="1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5" Sep 10 17:42:43.050128 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:43.050112 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5\": container with ID starting with 1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5 not found: ID does not exist" containerID="1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5" Sep 10 17:42:43.050171 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.050134 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5"} err="failed to get container status \"1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5\": rpc error: code = NotFound desc = could not find container \"1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5\": container with ID starting with 1a46cf93eb33f6a841905a0950c82f47cceb7b2aa82f863407ce3731583f84d5 not found: ID does not exist" Sep 10 17:42:43.064071 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.064049 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv"] Sep 10 17:42:43.064334 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.064321 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" containerName="registry" Sep 10 17:42:43.064384 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.064336 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" containerName="registry" Sep 10 17:42:43.064384 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.064382 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" containerName="registry" Sep 10 17:42:43.069597 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.069582 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.072601 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.072577 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 17:42:43.072695 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.072629 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 17:42:43.072695 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.072683 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xfzb6\"" Sep 10 17:42:43.072810 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.072764 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 17:42:43.077548 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.077505 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz"] Sep 10 17:42:43.080802 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.080782 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bcb66bc9-ql8cz"] Sep 10 17:42:43.080879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.080806 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv"] Sep 10 17:42:43.080921 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.080899 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.083750 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.083729 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wkcg2\"" Sep 10 17:42:43.083847 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.083747 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 17:42:43.083847 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.083731 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 17:42:43.083847 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.083734 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 17:42:43.084240 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.084209 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7bcb66bc9-ql8cz"] Sep 10 17:42:43.094115 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.094092 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k9sl7"] Sep 10 17:42:43.098373 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.098354 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz"] Sep 10 17:42:43.098452 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.098437 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.100953 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.100936 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 17:42:43.101061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.100941 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5vtvp\"" Sep 10 17:42:43.101061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.101000 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 17:42:43.101254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.101240 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 17:42:43.104816 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.104773 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.123694 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dc6w\" (UniqueName: \"kubernetes.io/projected/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-kube-api-access-5dc6w\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.123694 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123696 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-sys\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.123818 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnb6f\" (UniqueName: \"kubernetes.io/projected/d248c12c-b9d4-458b-8182-48b0c12a3303-kube-api-access-hnb6f\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.123818 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.123818 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-wtmp\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.123818 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.123818 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123813 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.124013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123860 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.124013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-tls\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.124013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.124013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1d2fbff6-084b-4725-9b96-276231f3c073-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.124013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.123975 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d248c12c-b9d4-458b-8182-48b0c12a3303-metrics-client-ca\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.124013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124010 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-accelerators-collector-config\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.124312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4b4d\" (UniqueName: \"kubernetes.io/projected/1d2fbff6-084b-4725-9b96-276231f3c073-kube-api-access-l4b4d\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.124312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124058 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-textfile\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.124312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124078 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.124312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.124312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d2fbff6-084b-4725-9b96-276231f3c073-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.124312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.124138 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-root\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.180458 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.180437 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.224734 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-accelerators-collector-config\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.224842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4b4d\" (UniqueName: \"kubernetes.io/projected/1d2fbff6-084b-4725-9b96-276231f3c073-kube-api-access-l4b4d\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.224842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224756 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-textfile\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.224842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224778 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.225002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.225002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d2fbff6-084b-4725-9b96-276231f3c073-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.225002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.224996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-root\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dc6w\" (UniqueName: \"kubernetes.io/projected/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-kube-api-access-5dc6w\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.225159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-sys\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225084 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-textfile\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225101 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnb6f\" (UniqueName: \"kubernetes.io/projected/d248c12c-b9d4-458b-8182-48b0c12a3303-kube-api-access-hnb6f\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-root\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225146 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-wtmp\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225204 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225265 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225302 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-sys\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225326 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-tls\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225348 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-accelerators-collector-config\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225357 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1d2fbff6-084b-4725-9b96-276231f3c073-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.225479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d248c12c-b9d4-458b-8182-48b0c12a3303-metrics-client-ca\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.225968 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225760 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d2fbff6-084b-4725-9b96-276231f3c073-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.225968 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:43.225865 2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 10 17:42:43.225968 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:43.225934 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-tls podName:75f70879-f9cd-4ea4-b9c8-518e071f5c5b nodeName:}" failed. No retries permitted until 2026-09-10 17:42:43.725913923 +0000 UTC m=+92.607639054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-g6zvv" (UID: "75f70879-f9cd-4ea4-b9c8-518e071f5c5b") : secret "openshift-state-metrics-tls" not found Sep 10 17:42:43.225968 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225942 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1d2fbff6-084b-4725-9b96-276231f3c073-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.226180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.225986 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d248c12c-b9d4-458b-8182-48b0c12a3303-metrics-client-ca\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.226180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.226067 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.226180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.226076 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-wtmp\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.226723 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.226689 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.227587 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.227564 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.227683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.227669 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.227940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.227919 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d2fbff6-084b-4725-9b96-276231f3c073-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.227940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.227939 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.228351 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.228336 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d248c12c-b9d4-458b-8182-48b0c12a3303-node-exporter-tls\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.233490 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.233472 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dc6w\" (UniqueName: \"kubernetes.io/projected/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-kube-api-access-5dc6w\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.237088 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.237071 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnb6f\" (UniqueName: \"kubernetes.io/projected/d248c12c-b9d4-458b-8182-48b0c12a3303-kube-api-access-hnb6f\") pod \"node-exporter-k9sl7\" (UID: \"d248c12c-b9d4-458b-8182-48b0c12a3303\") " pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.237359 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.237342 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4b4d\" (UniqueName: \"kubernetes.io/projected/1d2fbff6-084b-4725-9b96-276231f3c073-kube-api-access-l4b4d\") pod \"kube-state-metrics-74cb57df7-mwkjz\" (UID: \"1d2fbff6-084b-4725-9b96-276231f3c073\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.280115 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.280095 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.382624 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.382566 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.390396 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.390368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" Sep 10 17:42:43.406621 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.406598 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k9sl7" Sep 10 17:42:43.417573 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:43.417538 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd248c12c_b9d4_458b_8182_48b0c12a3303.slice/crio-258a8bf44ff49c3f210ef9aa91a7bbbfe31451a64a62a72d591aa469ea2a6ec5 WatchSource:0}: Error finding container 258a8bf44ff49c3f210ef9aa91a7bbbfe31451a64a62a72d591aa469ea2a6ec5: Status 404 returned error can't find the container with id 258a8bf44ff49c3f210ef9aa91a7bbbfe31451a64a62a72d591aa469ea2a6ec5 Sep 10 17:42:43.481537 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.481509 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.592111 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.592070 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.622021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.621988 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f26f85db-bbe5-4cc7-a49e-1eb8268f24e9" path="/var/lib/kubelet/pods/f26f85db-bbe5-4cc7-a49e-1eb8268f24e9/volumes" Sep 10 17:42:43.670533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.670500 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz"] Sep 10 17:42:43.672661 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:43.672605 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2fbff6_084b_4725_9b96_276231f3c073.slice/crio-77c646909e4b29ccb7d08ead688ec11860fd62571b92641fdd877dea83e9fcab WatchSource:0}: Error finding container 77c646909e4b29ccb7d08ead688ec11860fd62571b92641fdd877dea83e9fcab: Status 404 returned error can't find the container with id 77c646909e4b29ccb7d08ead688ec11860fd62571b92641fdd877dea83e9fcab Sep 10 17:42:43.675056 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.675030 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.729554 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.729528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.731703 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.731686 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75f70879-f9cd-4ea4-b9c8-518e071f5c5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-g6zvv\" (UID: \"75f70879-f9cd-4ea4-b9c8-518e071f5c5b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:43.979075 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:43.978995 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" Sep 10 17:42:44.005617 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.005577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.057524 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.057493 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k9sl7" event={"ID":"d248c12c-b9d4-458b-8182-48b0c12a3303","Type":"ContainerStarted","Data":"258a8bf44ff49c3f210ef9aa91a7bbbfe31451a64a62a72d591aa469ea2a6ec5"} Sep 10 17:42:44.063536 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.063368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" event={"ID":"1d2fbff6-084b-4725-9b96-276231f3c073","Type":"ContainerStarted","Data":"77c646909e4b29ccb7d08ead688ec11860fd62571b92641fdd877dea83e9fcab"} Sep 10 17:42:44.090886 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.090846 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.091637 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.091611 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:42:44.096038 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.096014 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.099043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.099019 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 17:42:44.099390 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.099372 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 17:42:44.099589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.099573 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 17:42:44.099802 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.099763 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 17:42:44.099802 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.099798 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 17:42:44.099935 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.099812 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 17:42:44.100107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.100081 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n79v8\"" Sep 10 17:42:44.100273 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.100255 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 17:42:44.100357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.100274 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 17:42:44.100357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.100328 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 17:42:44.112559 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.112538 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:42:44.132671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132685 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132725 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132799 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-config-out\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132827 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132860 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.132975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.132954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9zjb\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-kube-api-access-r9zjb\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.133165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.133004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.133165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.133057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.133165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.133084 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.133165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.133107 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-web-config\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.133165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.133149 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.153426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.153389 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.206969 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.206821 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.234107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-config-out\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234075 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234166 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9zjb\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-kube-api-access-r9zjb\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234273 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234309 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-web-config\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234727 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234354 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234727 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234371 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.234727 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.234401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.235808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.235674 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.236573 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.236343 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.237628 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.237605 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.237742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.237716 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-config-out\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.237822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.237797 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.240011 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.239945 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.240011 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.239967 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-web-config\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.241221 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.241176 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.241349 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.241319 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.242073 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.242037 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.242582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.242548 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.243513 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.243493 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.244910 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.244856 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9zjb\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-kube-api-access-r9zjb\") pod \"alertmanager-main-0\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.268091 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.268011 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv"] Sep 10 17:42:44.269956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.269928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.361047 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.361012 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:44.361255 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.361088 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:44.366917 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.366892 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:44.381022 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:44.380988 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f70879_f9cd_4ea4_b9c8_518e071f5c5b.slice/crio-87a2c2bcf9daec1accdfe97d97c659f3d1ae2e5b9bd20913d4c5babe0ff43888 WatchSource:0}: Error finding container 87a2c2bcf9daec1accdfe97d97c659f3d1ae2e5b9bd20913d4c5babe0ff43888: Status 404 returned error can't find the container with id 87a2c2bcf9daec1accdfe97d97c659f3d1ae2e5b9bd20913d4c5babe0ff43888 Sep 10 17:42:44.407734 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.407707 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:42:44.478992 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.478958 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.579287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.579211 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:44.730053 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:44.730023 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:42:44.946341 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:44.946305 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a35953_20a7_42ca_9095_b89c17510a94.slice/crio-bb719e4d8b1867ce20a9e390840e45254a524f5906e976caaa40c0a089e3310b WatchSource:0}: Error finding container bb719e4d8b1867ce20a9e390840e45254a524f5906e976caaa40c0a089e3310b: Status 404 returned error can't find the container with id bb719e4d8b1867ce20a9e390840e45254a524f5906e976caaa40c0a089e3310b Sep 10 17:42:45.075641 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.075612 2565 generic.go:356] "Generic (PLEG): container finished" podID="d248c12c-b9d4-458b-8182-48b0c12a3303" containerID="ffe4e91752156c1d31454a75ff25489af95d543ebffabbfea1101db77e55c904" exitCode=0 Sep 10 17:42:45.075758 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.075699 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k9sl7" event={"ID":"d248c12c-b9d4-458b-8182-48b0c12a3303","Type":"ContainerDied","Data":"ffe4e91752156c1d31454a75ff25489af95d543ebffabbfea1101db77e55c904"} Sep 10 17:42:45.078626 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.078549 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" event={"ID":"1d2fbff6-084b-4725-9b96-276231f3c073","Type":"ContainerStarted","Data":"6bddce848e167967e1413c65ae746ba7f2682c04f3848ae0a4b9d22d46f9a41e"} Sep 10 17:42:45.080576 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.080552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"bb719e4d8b1867ce20a9e390840e45254a524f5906e976caaa40c0a089e3310b"} Sep 10 17:42:45.083028 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.083006 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" event={"ID":"75f70879-f9cd-4ea4-b9c8-518e071f5c5b","Type":"ContainerStarted","Data":"e56a6d9f836402d88de78d425c37326a759e8ffe439565f66e54695b3ee748a7"} Sep 10 17:42:45.083137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.083060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" event={"ID":"75f70879-f9cd-4ea4-b9c8-518e071f5c5b","Type":"ContainerStarted","Data":"72675ea06f66f9857e9cc0f75339662d8c38bd202c74ffae9cf42422290058f1"} Sep 10 17:42:45.083137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.083092 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" event={"ID":"75f70879-f9cd-4ea4-b9c8-518e071f5c5b","Type":"ContainerStarted","Data":"87a2c2bcf9daec1accdfe97d97c659f3d1ae2e5b9bd20913d4c5babe0ff43888"} Sep 10 17:42:45.087444 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.087410 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:42:45.152205 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.151518 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8474f697f7-twrsz"] Sep 10 17:42:45.317441 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.317358 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:45.418866 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.418797 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:45.519080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:45.518994 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.010362 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.010338 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.083146 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.083105 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.089687 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.089656 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" event={"ID":"75f70879-f9cd-4ea4-b9c8-518e071f5c5b","Type":"ContainerStarted","Data":"4481a50f1cd4f2fe04ba422627be28e827123f0454811fdcc1e1c8d2f47de912"} Sep 10 17:42:46.092066 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.092040 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k9sl7" event={"ID":"d248c12c-b9d4-458b-8182-48b0c12a3303","Type":"ContainerStarted","Data":"4a0c667d53a19dd4efe28eebaf7a18917272e4c1f3008e0283631517310fb6af"} Sep 10 17:42:46.092154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.092093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k9sl7" event={"ID":"d248c12c-b9d4-458b-8182-48b0c12a3303","Type":"ContainerStarted","Data":"8196f114f6b394778d8ed739851e54aafc374977a3f10b80a4a82597b2a4eb0f"} Sep 10 17:42:46.094034 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.094012 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" event={"ID":"1d2fbff6-084b-4725-9b96-276231f3c073","Type":"ContainerStarted","Data":"d1d2da2ced9ffd150525088a0990e4b146c6930e109a04ca7d7c53ee99ef39fe"} Sep 10 17:42:46.094141 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.094090 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" event={"ID":"1d2fbff6-084b-4725-9b96-276231f3c073","Type":"ContainerStarted","Data":"27df0941b91acccdedd15d2ac3f6a3829e75b90db6fd75a422b47ccc79bd32ed"} Sep 10 17:42:46.095779 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.095758 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798"} Sep 10 17:42:46.113240 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.113179 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-g6zvv" podStartSLOduration=1.662623374 podStartE2EDuration="3.113166606s" totalImagesPullingTime="1.450543232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:46.112845235 +0000 UTC m=+94.994570377" watchObservedRunningTime="2026-09-10 17:42:46.113166606 +0000 UTC m=+94.994891747" Sep 10 17:42:46.160970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.160929 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k9sl7" podStartSLOduration=2.074300712 podStartE2EDuration="3.160916675s" totalImagesPullingTime="1.016960891s" totalInitContainerRuntime="69.655072ms" isStatefulPod=true podCreationTimestamp="2026-09-10 17:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:46.159341823 +0000 UTC m=+95.041066965" watchObservedRunningTime="2026-09-10 17:42:46.160916675 +0000 UTC m=+95.042641815" Sep 10 17:42:46.179266 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.179197 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-mwkjz" podStartSLOduration=1.863211221 podStartE2EDuration="3.179184532s" totalImagesPullingTime="1.315973311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:46.178082586 +0000 UTC m=+95.059807728" watchObservedRunningTime="2026-09-10 17:42:46.179184532 +0000 UTC m=+95.060909673" Sep 10 17:42:46.313696 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.313623 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.545531 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.545497 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.601072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:46.601045 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:47.104129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.104091 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798" exitCode=0 Sep 10 17:42:47.104569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.104260 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798"} Sep 10 17:42:47.816621 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.816592 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:47.821455 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.821429 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6"] Sep 10 17:42:47.823560 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.823538 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:47.826212 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.826194 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vzj57\"" Sep 10 17:42:47.826575 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.826559 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 17:42:47.832772 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.832752 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6"] Sep 10 17:42:47.872353 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.872326 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee3970c1-46d6-4f00-b183-916b1778a572-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:47.872443 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.872394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwmk6\" (UniqueName: \"kubernetes.io/projected/ee3970c1-46d6-4f00-b183-916b1778a572-kube-api-access-pwmk6\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:47.873198 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.873178 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:47.973178 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.973152 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee3970c1-46d6-4f00-b183-916b1778a572-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:47.973278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.973196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwmk6\" (UniqueName: \"kubernetes.io/projected/ee3970c1-46d6-4f00-b183-916b1778a572-kube-api-access-pwmk6\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:47.973330 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:47.973299 2565 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 17:42:47.973364 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:42:47.973353 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee3970c1-46d6-4f00-b183-916b1778a572-monitoring-plugin-cert podName:ee3970c1-46d6-4f00-b183-916b1778a572 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:48.473337556 +0000 UTC m=+97.355062706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ee3970c1-46d6-4f00-b183-916b1778a572-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-jkjb6" (UID: "ee3970c1-46d6-4f00-b183-916b1778a572") : secret "monitoring-plugin-cert" not found Sep 10 17:42:47.984043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:47.984024 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwmk6\" (UniqueName: \"kubernetes.io/projected/ee3970c1-46d6-4f00-b183-916b1778a572-kube-api-access-pwmk6\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:48.087957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.087892 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b68d7f8f4-g87sl"] Sep 10 17:42:48.089984 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.089968 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.107104 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.107084 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b68d7f8f4-g87sl"] Sep 10 17:42:48.112415 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.112392 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231"} Sep 10 17:42:48.112488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.112456 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2"} Sep 10 17:42:48.112549 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.112494 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1"} Sep 10 17:42:48.112549 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.112519 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf"} Sep 10 17:42:48.112549 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.112543 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38"} Sep 10 17:42:48.153396 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.153366 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:48.175136 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-oauth-serving-cert\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.175266 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175151 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhqbv\" (UniqueName: \"kubernetes.io/projected/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-kube-api-access-rhqbv\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.175266 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175198 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-service-ca\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.175353 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-oauth-config\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.175353 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-trusted-ca-bundle\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.175416 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175363 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-config\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.175416 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.175401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-serving-cert\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276047 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276016 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-oauth-config\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276047 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-trusted-ca-bundle\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276068 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-config\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-serving-cert\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-oauth-serving-cert\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhqbv\" (UniqueName: \"kubernetes.io/projected/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-kube-api-access-rhqbv\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-service-ca\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.276953 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276918 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-config\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.277061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276954 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-trusted-ca-bundle\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.277061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276976 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-oauth-serving-cert\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.277061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.276993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-service-ca\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.278964 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.278938 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-oauth-config\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.279092 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.279070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-serving-cert\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.287379 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.287352 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhqbv\" (UniqueName: \"kubernetes.io/projected/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-kube-api-access-rhqbv\") pod \"console-5b68d7f8f4-g87sl\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.317055 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.317030 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p"] Sep 10 17:42:48.320531 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.320510 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.320944 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.320917 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:48.326300 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.326274 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 17:42:48.326413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.326276 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 17:42:48.326413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.326279 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 17:42:48.326618 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.326602 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7dph7\"" Sep 10 17:42:48.326725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.326603 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 17:42:48.326857 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.326833 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 17:42:48.333055 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.333035 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 17:42:48.344620 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.339259 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p"] Sep 10 17:42:48.377407 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377367 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-serving-certs-ca-bundle\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377529 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377408 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-secret-telemeter-client\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377529 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377487 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltl9k\" (UniqueName: \"kubernetes.io/projected/9d6784c3-02a8-4cea-ac8c-b52a40d91477-kube-api-access-ltl9k\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377635 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-telemeter-client-tls\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377635 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377614 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377758 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-federate-client-tls\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377758 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377745 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.377859 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.377842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-metrics-client-ca\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.379867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.379843 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:48.398741 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.398721 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:48.478896 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.478866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-metrics-client-ca\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.478903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-serving-certs-ca-bundle\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.478922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-secret-telemeter-client\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.478946 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee3970c1-46d6-4f00-b183-916b1778a572-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.478964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltl9k\" (UniqueName: \"kubernetes.io/projected/9d6784c3-02a8-4cea-ac8c-b52a40d91477-kube-api-access-ltl9k\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.478985 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-telemeter-client-tls\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.479012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.479036 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-federate-client-tls\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.479430 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.479060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.480197 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.480045 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.480389 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.480361 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-serving-certs-ca-bundle\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.480569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.480542 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6784c3-02a8-4cea-ac8c-b52a40d91477-metrics-client-ca\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.482477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.482448 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee3970c1-46d6-4f00-b183-916b1778a572-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jkjb6\" (UID: \"ee3970c1-46d6-4f00-b183-916b1778a572\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:48.482700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.482654 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-secret-telemeter-client\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.482867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.482845 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.483118 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.483073 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-telemeter-client-tls\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.483640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.483619 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6784c3-02a8-4cea-ac8c-b52a40d91477-federate-client-tls\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.488496 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.488474 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltl9k\" (UniqueName: \"kubernetes.io/projected/9d6784c3-02a8-4cea-ac8c-b52a40d91477-kube-api-access-ltl9k\") pod \"telemeter-client-5c8b855fcd-g9k2p\" (UID: \"9d6784c3-02a8-4cea-ac8c-b52a40d91477\") " pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.631745 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.631717 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" Sep 10 17:42:48.677794 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.677725 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b68d7f8f4-g87sl"] Sep 10 17:42:48.683016 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:48.682965 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dda579_9a30_4a7b_ae51_3c0b0b2b8a63.slice/crio-6ecef5f273ce95bb58d212ecba6599fcd9f20ca60df15d5827e0abf3715e5be9 WatchSource:0}: Error finding container 6ecef5f273ce95bb58d212ecba6599fcd9f20ca60df15d5827e0abf3715e5be9: Status 404 returned error can't find the container with id 6ecef5f273ce95bb58d212ecba6599fcd9f20ca60df15d5827e0abf3715e5be9 Sep 10 17:42:48.732910 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.732889 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:48.913674 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:48.913596 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p"] Sep 10 17:42:48.917032 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:48.917004 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6784c3_02a8_4cea_ac8c_b52a40d91477.slice/crio-2fb4ae980a8e407a97b327e536876fad5033bf08fd5d31d61e3460918436b3f8 WatchSource:0}: Error finding container 2fb4ae980a8e407a97b327e536876fad5033bf08fd5d31d61e3460918436b3f8: Status 404 returned error can't find the container with id 2fb4ae980a8e407a97b327e536876fad5033bf08fd5d31d61e3460918436b3f8 Sep 10 17:42:49.005648 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.005628 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6"] Sep 10 17:42:49.007823 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:49.007799 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3970c1_46d6_4f00_b183_916b1778a572.slice/crio-242c6348b5e7823c6dc49c60674a0d05003bf27ce881c44c5f0530e431ef72e1 WatchSource:0}: Error finding container 242c6348b5e7823c6dc49c60674a0d05003bf27ce881c44c5f0530e431ef72e1: Status 404 returned error can't find the container with id 242c6348b5e7823c6dc49c60674a0d05003bf27ce881c44c5f0530e431ef72e1 Sep 10 17:42:49.122263 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.122220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" event={"ID":"ee3970c1-46d6-4f00-b183-916b1778a572","Type":"ContainerStarted","Data":"242c6348b5e7823c6dc49c60674a0d05003bf27ce881c44c5f0530e431ef72e1"} Sep 10 17:42:49.123407 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.123382 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" event={"ID":"9d6784c3-02a8-4cea-ac8c-b52a40d91477","Type":"ContainerStarted","Data":"2fb4ae980a8e407a97b327e536876fad5033bf08fd5d31d61e3460918436b3f8"} Sep 10 17:42:49.126434 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.126405 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerStarted","Data":"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f"} Sep 10 17:42:49.127861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.127841 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b68d7f8f4-g87sl" event={"ID":"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63","Type":"ContainerStarted","Data":"7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691"} Sep 10 17:42:49.127964 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.127873 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b68d7f8f4-g87sl" event={"ID":"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63","Type":"ContainerStarted","Data":"6ecef5f273ce95bb58d212ecba6599fcd9f20ca60df15d5827e0abf3715e5be9"} Sep 10 17:42:49.164259 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.164162 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.892733259 podStartE2EDuration="5.164148524s" totalImagesPullingTime="3.253857658s" totalInitContainerRuntime="17.557607ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:49.162024615 +0000 UTC m=+98.043749756" watchObservedRunningTime="2026-09-10 17:42:49.164148524 +0000 UTC m=+98.045873666" Sep 10 17:42:49.182991 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.182947 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b68d7f8f4-g87sl" podStartSLOduration=1.182933889 podStartE2EDuration="1.182933889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:49.181741924 +0000 UTC m=+98.063467067" watchObservedRunningTime="2026-09-10 17:42:49.182933889 +0000 UTC m=+98.064659087" Sep 10 17:42:49.266531 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:49.266466 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:51.141669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.141639 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" event={"ID":"9d6784c3-02a8-4cea-ac8c-b52a40d91477","Type":"ContainerStarted","Data":"c7ef858f60e27466ce66a57022af10a7a5a94897579ef21979dd8928d39db2b6"} Sep 10 17:42:51.141986 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.141708 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" event={"ID":"9d6784c3-02a8-4cea-ac8c-b52a40d91477","Type":"ContainerStarted","Data":"9c435b790f678ec27c632f5793b06080c8cc773af31c803d53f4e7026239d564"} Sep 10 17:42:51.143180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.143157 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" event={"ID":"ee3970c1-46d6-4f00-b183-916b1778a572","Type":"ContainerStarted","Data":"f21f764a78b584d3a62894e79555edff43b70a588cb8fb01533c5bdd68edd954"} Sep 10 17:42:51.143432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.143410 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:51.147791 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.147772 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" Sep 10 17:42:51.161291 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.161252 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jkjb6" podStartSLOduration=2.254437977 podStartE2EDuration="4.161212699s" totalImagesPullingTime="1.906774722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:51.160406359 +0000 UTC m=+100.042131501" watchObservedRunningTime="2026-09-10 17:42:51.161212699 +0000 UTC m=+100.042937840" Sep 10 17:42:51.650499 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.650466 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:51.751520 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.751482 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:51.815782 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:51.815757 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:52.149508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.149468 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" event={"ID":"9d6784c3-02a8-4cea-ac8c-b52a40d91477","Type":"ContainerStarted","Data":"4b09facb9a7bd9a4eb9227fd283e86df6b7019ba956d43e71cc39faeb513f2f1"} Sep 10 17:42:52.174572 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.174526 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c8b855fcd-g9k2p" podStartSLOduration=2.131429391 podStartE2EDuration="4.174509874s" totalImagesPullingTime="2.043080483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:52.172367842 +0000 UTC m=+101.054092983" watchObservedRunningTime="2026-09-10 17:42:52.174509874 +0000 UTC m=+101.056235014" Sep 10 17:42:52.274435 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.274406 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b68d7f8f4-g87sl"] Sep 10 17:42:52.307771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.307741 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-db77d758-w2g4f"] Sep 10 17:42:52.348483 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.348455 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-db77d758-w2g4f"] Sep 10 17:42:52.348610 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.348538 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416101 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416013 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-serving-cert\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416101 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-trusted-ca-bundle\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416101 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-oauth-config\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k8tr\" (UniqueName: \"kubernetes.io/projected/90b4d527-70b7-41a8-aceb-e069264422f2-kube-api-access-5k8tr\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416253 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-console-config\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-service-ca\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.416432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.416319 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-oauth-serving-cert\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517141 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-serving-cert\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517151 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-trusted-ca-bundle\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-oauth-config\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k8tr\" (UniqueName: \"kubernetes.io/projected/90b4d527-70b7-41a8-aceb-e069264422f2-kube-api-access-5k8tr\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-console-config\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517268 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-service-ca\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.517283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.517283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-oauth-serving-cert\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.518046 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.518015 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-console-config\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.518147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.518044 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-service-ca\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.518147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.518057 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-oauth-serving-cert\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.518289 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.518190 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-trusted-ca-bundle\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.519761 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.519741 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-serving-cert\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.520008 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.519985 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-oauth-config\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.525379 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.525362 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k8tr\" (UniqueName: \"kubernetes.io/projected/90b4d527-70b7-41a8-aceb-e069264422f2-kube-api-access-5k8tr\") pod \"console-db77d758-w2g4f\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.657509 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.657478 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:42:52.959494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:52.959469 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-db77d758-w2g4f"] Sep 10 17:42:52.961782 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:42:52.961753 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b4d527_70b7_41a8_aceb_e069264422f2.slice/crio-beee7306ed7910635baa46f54da48f55eb7cb82751ba2a8fbab91e4bb6a52078 WatchSource:0}: Error finding container beee7306ed7910635baa46f54da48f55eb7cb82751ba2a8fbab91e4bb6a52078: Status 404 returned error can't find the container with id beee7306ed7910635baa46f54da48f55eb7cb82751ba2a8fbab91e4bb6a52078 Sep 10 17:42:53.156121 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:53.156082 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db77d758-w2g4f" event={"ID":"90b4d527-70b7-41a8-aceb-e069264422f2","Type":"ContainerStarted","Data":"b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3"} Sep 10 17:42:53.156491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:53.156143 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db77d758-w2g4f" event={"ID":"90b4d527-70b7-41a8-aceb-e069264422f2","Type":"ContainerStarted","Data":"beee7306ed7910635baa46f54da48f55eb7cb82751ba2a8fbab91e4bb6a52078"} Sep 10 17:42:53.173808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:53.173768 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-db77d758-w2g4f" podStartSLOduration=1.173754919 podStartE2EDuration="1.173754919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:53.17355702 +0000 UTC m=+102.055282163" watchObservedRunningTime="2026-09-10 17:42:53.173754919 +0000 UTC m=+102.055480059" Sep 10 17:42:53.964977 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:53.964948 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-prwhp" Sep 10 17:42:54.124167 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:54.124130 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:54.227429 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:54.227362 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:58.399278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:58.399243 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:42:59.149086 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:59.149047 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:59.257151 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:42:59.257112 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:02.658265 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:02.658208 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:43:02.658671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:02.658338 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:43:02.663140 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:02.663120 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:43:03.204120 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:03.204094 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:43:03.263241 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:03.263197 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5755485f76-n4wrz"] Sep 10 17:43:03.390631 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:03.390602 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:03.444385 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:03.444337 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:03.506252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:03.506133 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:10.184297 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.184261 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8474f697f7-twrsz" podUID="72611a31-79ad-40dd-a69b-e83b1f12a30b" containerName="console" containerID="cri-o://71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017" gracePeriod=15 Sep 10 17:43:10.424618 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.424594 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:43:10.563432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563360 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-config\") pod \"72611a31-79ad-40dd-a69b-e83b1f12a30b\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " Sep 10 17:43:10.563432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563403 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87ts8\" (UniqueName: \"kubernetes.io/projected/72611a31-79ad-40dd-a69b-e83b1f12a30b-kube-api-access-87ts8\") pod \"72611a31-79ad-40dd-a69b-e83b1f12a30b\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " Sep 10 17:43:10.563597 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563435 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-serving-cert\") pod \"72611a31-79ad-40dd-a69b-e83b1f12a30b\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " Sep 10 17:43:10.563597 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563458 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-oauth-config\") pod \"72611a31-79ad-40dd-a69b-e83b1f12a30b\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " Sep 10 17:43:10.563597 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563545 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-service-ca\") pod \"72611a31-79ad-40dd-a69b-e83b1f12a30b\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " Sep 10 17:43:10.563597 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563588 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-oauth-serving-cert\") pod \"72611a31-79ad-40dd-a69b-e83b1f12a30b\" (UID: \"72611a31-79ad-40dd-a69b-e83b1f12a30b\") " Sep 10 17:43:10.563872 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563839 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-config" (OuterVolumeSpecName: "console-config") pod "72611a31-79ad-40dd-a69b-e83b1f12a30b" (UID: "72611a31-79ad-40dd-a69b-e83b1f12a30b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:10.563992 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.563897 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-service-ca" (OuterVolumeSpecName: "service-ca") pod "72611a31-79ad-40dd-a69b-e83b1f12a30b" (UID: "72611a31-79ad-40dd-a69b-e83b1f12a30b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:10.564074 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.564042 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "72611a31-79ad-40dd-a69b-e83b1f12a30b" (UID: "72611a31-79ad-40dd-a69b-e83b1f12a30b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:10.565731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.565710 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "72611a31-79ad-40dd-a69b-e83b1f12a30b" (UID: "72611a31-79ad-40dd-a69b-e83b1f12a30b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:10.565819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.565731 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72611a31-79ad-40dd-a69b-e83b1f12a30b-kube-api-access-87ts8" (OuterVolumeSpecName: "kube-api-access-87ts8") pod "72611a31-79ad-40dd-a69b-e83b1f12a30b" (UID: "72611a31-79ad-40dd-a69b-e83b1f12a30b"). InnerVolumeSpecName "kube-api-access-87ts8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:43:10.565819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.565775 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "72611a31-79ad-40dd-a69b-e83b1f12a30b" (UID: "72611a31-79ad-40dd-a69b-e83b1f12a30b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:10.665263 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.665209 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:10.665263 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.665259 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-oauth-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:10.665418 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.665274 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:10.665418 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.665288 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-oauth-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:10.665418 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.665300 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72611a31-79ad-40dd-a69b-e83b1f12a30b-console-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:10.665418 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:10.665312 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87ts8\" (UniqueName: \"kubernetes.io/projected/72611a31-79ad-40dd-a69b-e83b1f12a30b-kube-api-access-87ts8\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:11.237332 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.237291 2565 generic.go:356] "Generic (PLEG): container finished" podID="72611a31-79ad-40dd-a69b-e83b1f12a30b" containerID="71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017" exitCode=0 Sep 10 17:43:11.237710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.237342 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474f697f7-twrsz" Sep 10 17:43:11.237710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.237373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474f697f7-twrsz" event={"ID":"72611a31-79ad-40dd-a69b-e83b1f12a30b","Type":"ContainerDied","Data":"71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017"} Sep 10 17:43:11.237710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.237425 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474f697f7-twrsz" event={"ID":"72611a31-79ad-40dd-a69b-e83b1f12a30b","Type":"ContainerDied","Data":"8d854aa339263e9bc68b6fb5d4597d4c388ab10ada6710d421d4b4958c1447ff"} Sep 10 17:43:11.237710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.237445 2565 scope.go:117] "RemoveContainer" containerID="71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017" Sep 10 17:43:11.248270 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.248254 2565 scope.go:117] "RemoveContainer" containerID="71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017" Sep 10 17:43:11.248534 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:43:11.248514 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017\": container with ID starting with 71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017 not found: ID does not exist" containerID="71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017" Sep 10 17:43:11.248594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.248545 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017"} err="failed to get container status \"71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017\": rpc error: code = NotFound desc = could not find container \"71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017\": container with ID starting with 71886bed97502e3db8c50456c8b061e6a2557c72e0ccc06d20c1f7d4140f8017 not found: ID does not exist" Sep 10 17:43:11.262798 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.262775 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8474f697f7-twrsz"] Sep 10 17:43:11.264803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.264773 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8474f697f7-twrsz"] Sep 10 17:43:11.621290 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:11.621262 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72611a31-79ad-40dd-a69b-e83b1f12a30b" path="/var/lib/kubelet/pods/72611a31-79ad-40dd-a69b-e83b1f12a30b/volumes" Sep 10 17:43:14.139621 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:14.139587 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:14.192605 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:14.192572 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:17.294206 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.294168 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b68d7f8f4-g87sl" podUID="32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" containerName="console" containerID="cri-o://7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691" gracePeriod=15 Sep 10 17:43:17.535372 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.535346 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:43:17.623822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623758 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-oauth-serving-cert\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.623822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623798 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhqbv\" (UniqueName: \"kubernetes.io/projected/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-kube-api-access-rhqbv\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.624022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623828 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-service-ca\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.624022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623849 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-serving-cert\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.624022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623868 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-oauth-config\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.624022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623893 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-trusted-ca-bundle\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.624022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.623935 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-config\") pod \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\" (UID: \"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63\") " Sep 10 17:43:17.624297 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.624138 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:17.624358 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.624339 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-service-ca" (OuterVolumeSpecName: "service-ca") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:17.624439 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.624417 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-config" (OuterVolumeSpecName: "console-config") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:17.624483 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.624429 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:17.626076 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.626050 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:17.626150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.626077 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:17.626150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.626120 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-kube-api-access-rhqbv" (OuterVolumeSpecName: "kube-api-access-rhqbv") pod "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" (UID: "32dda579-9a30-4a7b-ae51-3c0b0b2b8a63"). InnerVolumeSpecName "kube-api-access-rhqbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:43:17.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725088 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhqbv\" (UniqueName: \"kubernetes.io/projected/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-kube-api-access-rhqbv\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:17.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725124 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:17.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725133 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:17.725127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725142 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-oauth-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:17.725362 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725151 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-trusted-ca-bundle\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:17.725362 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725160 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-console-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:17.725362 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:17.725168 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63-oauth-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:18.265610 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.265572 2565 generic.go:356] "Generic (PLEG): container finished" podID="32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" containerID="7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691" exitCode=0 Sep 10 17:43:18.265610 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.265611 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b68d7f8f4-g87sl" Sep 10 17:43:18.265815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.265641 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b68d7f8f4-g87sl" event={"ID":"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63","Type":"ContainerDied","Data":"7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691"} Sep 10 17:43:18.265815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.265682 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b68d7f8f4-g87sl" event={"ID":"32dda579-9a30-4a7b-ae51-3c0b0b2b8a63","Type":"ContainerDied","Data":"6ecef5f273ce95bb58d212ecba6599fcd9f20ca60df15d5827e0abf3715e5be9"} Sep 10 17:43:18.265815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.265702 2565 scope.go:117] "RemoveContainer" containerID="7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691" Sep 10 17:43:18.274182 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.274162 2565 scope.go:117] "RemoveContainer" containerID="7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691" Sep 10 17:43:18.274452 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:43:18.274432 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691\": container with ID starting with 7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691 not found: ID does not exist" containerID="7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691" Sep 10 17:43:18.274499 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.274461 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691"} err="failed to get container status \"7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691\": rpc error: code = NotFound desc = could not find container \"7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691\": container with ID starting with 7fce0178f1a0f0d29bcca0043e55d4bf3c370d172a3d7cd195c17376eeea2691 not found: ID does not exist" Sep 10 17:43:18.289136 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.289116 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b68d7f8f4-g87sl"] Sep 10 17:43:18.294826 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:18.294802 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b68d7f8f4-g87sl"] Sep 10 17:43:19.622265 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:19.622211 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" path="/var/lib/kubelet/pods/32dda579-9a30-4a7b-ae51-3c0b0b2b8a63/volumes" Sep 10 17:43:23.288536 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:23.288501 2565 generic.go:356] "Generic (PLEG): container finished" podID="e8ef2482-361b-49bd-8f72-a594191b49fc" containerID="7b611dcc6bd055e2ed60d0e655e20968f9ed12fcc711dc7160365a4ed295ad68" exitCode=0 Sep 10 17:43:23.288990 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:23.288572 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" event={"ID":"e8ef2482-361b-49bd-8f72-a594191b49fc","Type":"ContainerDied","Data":"7b611dcc6bd055e2ed60d0e655e20968f9ed12fcc711dc7160365a4ed295ad68"} Sep 10 17:43:23.288990 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:23.288877 2565 scope.go:117] "RemoveContainer" containerID="7b611dcc6bd055e2ed60d0e655e20968f9ed12fcc711dc7160365a4ed295ad68" Sep 10 17:43:24.294916 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:24.294878 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-624fp" event={"ID":"e8ef2482-361b-49bd-8f72-a594191b49fc","Type":"ContainerStarted","Data":"6a87a60ac6c873f4a1f925bf4a5ccb0efb08adb11d27132b7712c24d1e9ca31b"} Sep 10 17:43:28.282473 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.282415 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5755485f76-n4wrz" podUID="7835c975-e597-4338-b1e5-8cc5a1d3b279" containerName="console" containerID="cri-o://f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd" gracePeriod=15 Sep 10 17:43:28.312335 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.312308 2565 generic.go:356] "Generic (PLEG): container finished" podID="de119af3-a57d-41b9-a334-63c626cc1d27" containerID="fd3ef908f4dcf5fa4bb2309c0d0809a9c89f5455d1ca059105f9f5d311a3b12f" exitCode=0 Sep 10 17:43:28.312468 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.312358 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" event={"ID":"de119af3-a57d-41b9-a334-63c626cc1d27","Type":"ContainerDied","Data":"fd3ef908f4dcf5fa4bb2309c0d0809a9c89f5455d1ca059105f9f5d311a3b12f"} Sep 10 17:43:28.312655 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.312641 2565 scope.go:117] "RemoveContainer" containerID="fd3ef908f4dcf5fa4bb2309c0d0809a9c89f5455d1ca059105f9f5d311a3b12f" Sep 10 17:43:28.526280 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.526253 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:43:28.614399 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614361 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxvtd\" (UniqueName: \"kubernetes.io/projected/7835c975-e597-4338-b1e5-8cc5a1d3b279-kube-api-access-xxvtd\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614461 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-oauth-config\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614496 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-serving-cert\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614522 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-service-ca\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614572 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-trusted-ca-bundle\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614807 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614609 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-oauth-serving-cert\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614807 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614641 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-config\") pod \"7835c975-e597-4338-b1e5-8cc5a1d3b279\" (UID: \"7835c975-e597-4338-b1e5-8cc5a1d3b279\") " Sep 10 17:43:28.614980 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.614946 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-service-ca" (OuterVolumeSpecName: "service-ca") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:28.615038 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.615012 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:28.615238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.615202 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-config" (OuterVolumeSpecName: "console-config") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:28.615325 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.615253 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:28.616650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.616618 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:28.616746 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.616680 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:28.616746 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.616698 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7835c975-e597-4338-b1e5-8cc5a1d3b279-kube-api-access-xxvtd" (OuterVolumeSpecName: "kube-api-access-xxvtd") pod "7835c975-e597-4338-b1e5-8cc5a1d3b279" (UID: "7835c975-e597-4338-b1e5-8cc5a1d3b279"). InnerVolumeSpecName "kube-api-access-xxvtd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:43:28.716113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716077 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-oauth-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:28.716113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716112 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:28.716323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716124 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:28.716323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716132 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-trusted-ca-bundle\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:28.716323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716141 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-oauth-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:28.716323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716151 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7835c975-e597-4338-b1e5-8cc5a1d3b279-console-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:28.716323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:28.716159 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxvtd\" (UniqueName: \"kubernetes.io/projected/7835c975-e597-4338-b1e5-8cc5a1d3b279-kube-api-access-xxvtd\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:43:29.318209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.318168 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnqlv" event={"ID":"de119af3-a57d-41b9-a334-63c626cc1d27","Type":"ContainerStarted","Data":"5518d247f0e13d1d6daaec520646d07c6c25cfc7c5ed5e7b91f49c83cbfd9830"} Sep 10 17:43:29.319533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.319504 2565 generic.go:356] "Generic (PLEG): container finished" podID="7835c975-e597-4338-b1e5-8cc5a1d3b279" containerID="f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd" exitCode=0 Sep 10 17:43:29.319669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.319541 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5755485f76-n4wrz" Sep 10 17:43:29.319669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.319557 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5755485f76-n4wrz" event={"ID":"7835c975-e597-4338-b1e5-8cc5a1d3b279","Type":"ContainerDied","Data":"f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd"} Sep 10 17:43:29.319669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.319599 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5755485f76-n4wrz" event={"ID":"7835c975-e597-4338-b1e5-8cc5a1d3b279","Type":"ContainerDied","Data":"81bed16070a83fbdccb0f4af9c79decde506a93a7b1501bf2497d5c4f6ccf4f9"} Sep 10 17:43:29.319669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.319624 2565 scope.go:117] "RemoveContainer" containerID="f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd" Sep 10 17:43:29.331812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.331781 2565 scope.go:117] "RemoveContainer" containerID="f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd" Sep 10 17:43:29.332160 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:43:29.332139 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd\": container with ID starting with f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd not found: ID does not exist" containerID="f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd" Sep 10 17:43:29.332219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.332168 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd"} err="failed to get container status \"f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd\": rpc error: code = NotFound desc = could not find container \"f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd\": container with ID starting with f3e5bb68f8961c617dd09e141f6442d1161316699ccddb34f98a07dc25322abd not found: ID does not exist" Sep 10 17:43:29.362322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.362238 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5755485f76-n4wrz"] Sep 10 17:43:29.364446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.364378 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5755485f76-n4wrz"] Sep 10 17:43:29.621546 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:29.621466 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7835c975-e597-4338-b1e5-8cc5a1d3b279" path="/var/lib/kubelet/pods/7835c975-e597-4338-b1e5-8cc5a1d3b279/volumes" Sep 10 17:43:54.967853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:54.967764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:59.167961 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:59.167926 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:59.266837 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:43:59.266805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:03.400593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400555 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:44:03.401012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400857 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="alertmanager" containerID="cri-o://879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38" gracePeriod=120 Sep 10 17:44:03.401012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400903 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-web" containerID="cri-o://9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1" gracePeriod=120 Sep 10 17:44:03.401012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400913 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy" containerID="cri-o://43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2" gracePeriod=120 Sep 10 17:44:03.401012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400937 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="config-reloader" containerID="cri-o://a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf" gracePeriod=120 Sep 10 17:44:03.401012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400949 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="prom-label-proxy" containerID="cri-o://a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f" gracePeriod=120 Sep 10 17:44:03.401012 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:03.400988 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-metric" containerID="cri-o://bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231" gracePeriod=120 Sep 10 17:44:04.467141 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467110 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f" exitCode=0 Sep 10 17:44:04.467141 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467137 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2" exitCode=0 Sep 10 17:44:04.467141 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467144 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf" exitCode=0 Sep 10 17:44:04.467525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467150 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38" exitCode=0 Sep 10 17:44:04.467525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467176 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f"} Sep 10 17:44:04.467525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467283 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2"} Sep 10 17:44:04.467525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467313 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf"} Sep 10 17:44:04.467525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.467338 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38"} Sep 10 17:44:04.643718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.643695 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:04.797245 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797134 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-main-db\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797245 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797177 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-config-volume\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797245 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797203 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-main-tls\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797252 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-metrics-client-ca\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797269 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-web-config\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797289 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-config-out\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797309 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797420 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-tls-assets\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797475 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9zjb\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-kube-api-access-r9zjb\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797519 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797521 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-trusted-ca-bundle\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797564 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-cluster-tls-config\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797561 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:44:04.797874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797592 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.797874 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.797653 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:44:04.798098 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.798072 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:44:04.798260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.798209 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-web\") pod \"a4a35953-20a7-42ca-9095-b89c17510a94\" (UID: \"a4a35953-20a7-42ca-9095-b89c17510a94\") " Sep 10 17:44:04.798526 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.798502 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.798612 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.798532 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-alertmanager-main-db\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.798612 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.798547 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a35953-20a7-42ca-9095-b89c17510a94-metrics-client-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.800486 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.800459 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.801016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.800966 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.801016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.800966 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-config-out" (OuterVolumeSpecName: "config-out") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:44:04.801016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.801008 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-config-volume" (OuterVolumeSpecName: "config-volume") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.801214 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.801046 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.801356 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.801337 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:44:04.801522 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.801500 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-kube-api-access-r9zjb" (OuterVolumeSpecName: "kube-api-access-r9zjb") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "kube-api-access-r9zjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:44:04.801840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.801824 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.805252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.805145 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.810894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.810874 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-web-config" (OuterVolumeSpecName: "web-config") pod "a4a35953-20a7-42ca-9095-b89c17510a94" (UID: "a4a35953-20a7-42ca-9095-b89c17510a94"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:04.899458 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899424 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899458 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899453 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899458 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899464 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-main-tls\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899474 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-web-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899483 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a35953-20a7-42ca-9095-b89c17510a94-config-out\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899490 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899500 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-tls-assets\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899509 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9zjb\" (UniqueName: \"kubernetes.io/projected/a4a35953-20a7-42ca-9095-b89c17510a94-kube-api-access-r9zjb\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899517 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-cluster-tls-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:04.899679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:04.899525 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a35953-20a7-42ca-9095-b89c17510a94-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:05.474385 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474352 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231" exitCode=0 Sep 10 17:44:05.474385 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474379 2565 generic.go:356] "Generic (PLEG): container finished" podID="a4a35953-20a7-42ca-9095-b89c17510a94" containerID="9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1" exitCode=0 Sep 10 17:44:05.474733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474446 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.474733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474438 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231"} Sep 10 17:44:05.474733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474596 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1"} Sep 10 17:44:05.474733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a35953-20a7-42ca-9095-b89c17510a94","Type":"ContainerDied","Data":"bb719e4d8b1867ce20a9e390840e45254a524f5906e976caaa40c0a089e3310b"} Sep 10 17:44:05.474733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.474671 2565 scope.go:117] "RemoveContainer" containerID="a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f" Sep 10 17:44:05.483118 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.483097 2565 scope.go:117] "RemoveContainer" containerID="bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231" Sep 10 17:44:05.489970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.489955 2565 scope.go:117] "RemoveContainer" containerID="43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2" Sep 10 17:44:05.496454 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.496436 2565 scope.go:117] "RemoveContainer" containerID="9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1" Sep 10 17:44:05.500696 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.500678 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:44:05.503257 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.503238 2565 scope.go:117] "RemoveContainer" containerID="a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf" Sep 10 17:44:05.507759 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.507740 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:44:05.510438 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.510424 2565 scope.go:117] "RemoveContainer" containerID="879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38" Sep 10 17:44:05.517093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.516959 2565 scope.go:117] "RemoveContainer" containerID="b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798" Sep 10 17:44:05.523566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.523550 2565 scope.go:117] "RemoveContainer" containerID="a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f" Sep 10 17:44:05.524025 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.523847 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f\": container with ID starting with a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f not found: ID does not exist" containerID="a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f" Sep 10 17:44:05.524025 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.523990 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f"} err="failed to get container status \"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f\": rpc error: code = NotFound desc = could not find container \"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f\": container with ID starting with a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f not found: ID does not exist" Sep 10 17:44:05.524025 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524022 2565 scope.go:117] "RemoveContainer" containerID="bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231" Sep 10 17:44:05.524348 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.524331 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231\": container with ID starting with bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231 not found: ID does not exist" containerID="bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231" Sep 10 17:44:05.524402 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524353 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231"} err="failed to get container status \"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231\": rpc error: code = NotFound desc = could not find container \"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231\": container with ID starting with bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231 not found: ID does not exist" Sep 10 17:44:05.524402 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524370 2565 scope.go:117] "RemoveContainer" containerID="43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2" Sep 10 17:44:05.524625 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.524606 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2\": container with ID starting with 43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2 not found: ID does not exist" containerID="43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2" Sep 10 17:44:05.524684 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524631 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2"} err="failed to get container status \"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2\": rpc error: code = NotFound desc = could not find container \"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2\": container with ID starting with 43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2 not found: ID does not exist" Sep 10 17:44:05.524684 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524646 2565 scope.go:117] "RemoveContainer" containerID="9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1" Sep 10 17:44:05.524924 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.524906 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1\": container with ID starting with 9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1 not found: ID does not exist" containerID="9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1" Sep 10 17:44:05.524966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524929 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1"} err="failed to get container status \"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1\": rpc error: code = NotFound desc = could not find container \"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1\": container with ID starting with 9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1 not found: ID does not exist" Sep 10 17:44:05.524966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.524944 2565 scope.go:117] "RemoveContainer" containerID="a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf" Sep 10 17:44:05.525199 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.525181 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf\": container with ID starting with a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf not found: ID does not exist" containerID="a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf" Sep 10 17:44:05.525258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.525203 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf"} err="failed to get container status \"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf\": rpc error: code = NotFound desc = could not find container \"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf\": container with ID starting with a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf not found: ID does not exist" Sep 10 17:44:05.525258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.525217 2565 scope.go:117] "RemoveContainer" containerID="879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38" Sep 10 17:44:05.525475 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.525459 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38\": container with ID starting with 879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38 not found: ID does not exist" containerID="879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38" Sep 10 17:44:05.525513 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.525479 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38"} err="failed to get container status \"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38\": rpc error: code = NotFound desc = could not find container \"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38\": container with ID starting with 879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38 not found: ID does not exist" Sep 10 17:44:05.525513 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.525493 2565 scope.go:117] "RemoveContainer" containerID="b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798" Sep 10 17:44:05.525744 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:05.525727 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798\": container with ID starting with b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798 not found: ID does not exist" containerID="b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798" Sep 10 17:44:05.525792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.525747 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798"} err="failed to get container status \"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798\": rpc error: code = NotFound desc = could not find container \"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798\": container with ID starting with b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798 not found: ID does not exist" Sep 10 17:44:05.525792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.525760 2565 scope.go:117] "RemoveContainer" containerID="a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f" Sep 10 17:44:05.526023 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526004 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f"} err="failed to get container status \"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f\": rpc error: code = NotFound desc = could not find container \"a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f\": container with ID starting with a88703a164c743674ef03f882c2b192a7375e7937c0303500b79d1ef48d1317f not found: ID does not exist" Sep 10 17:44:05.526023 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526023 2565 scope.go:117] "RemoveContainer" containerID="bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231" Sep 10 17:44:05.526253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526218 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231"} err="failed to get container status \"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231\": rpc error: code = NotFound desc = could not find container \"bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231\": container with ID starting with bfc623c79b6db4f6d10acb9d765dff9415be7e479504dd017516401490088231 not found: ID does not exist" Sep 10 17:44:05.526303 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526254 2565 scope.go:117] "RemoveContainer" containerID="43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2" Sep 10 17:44:05.526489 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526471 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2"} err="failed to get container status \"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2\": rpc error: code = NotFound desc = could not find container \"43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2\": container with ID starting with 43a842b06548a41e70eb206624cbb85c86b2dc192bdf3d260512389c4bb80be2 not found: ID does not exist" Sep 10 17:44:05.526539 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526493 2565 scope.go:117] "RemoveContainer" containerID="9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1" Sep 10 17:44:05.526703 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526688 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1"} err="failed to get container status \"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1\": rpc error: code = NotFound desc = could not find container \"9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1\": container with ID starting with 9541973ae6ba88467c4e7a369e58f462731d6682e95a2e5fd4622574cfc4a2a1 not found: ID does not exist" Sep 10 17:44:05.526747 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526703 2565 scope.go:117] "RemoveContainer" containerID="a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf" Sep 10 17:44:05.526929 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526911 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf"} err="failed to get container status \"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf\": rpc error: code = NotFound desc = could not find container \"a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf\": container with ID starting with a503cc53c39e39f75b52a2ab2e11d0df7d3e01c5fc973782ac458ed83abed2cf not found: ID does not exist" Sep 10 17:44:05.526971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.526930 2565 scope.go:117] "RemoveContainer" containerID="879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38" Sep 10 17:44:05.527135 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.527120 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38"} err="failed to get container status \"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38\": rpc error: code = NotFound desc = could not find container \"879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38\": container with ID starting with 879589b53d28fb3171366f031f7c0beb83fc5f97ff8f77e3f63186c50f69fb38 not found: ID does not exist" Sep 10 17:44:05.527171 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.527135 2565 scope.go:117] "RemoveContainer" containerID="b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798" Sep 10 17:44:05.527362 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.527342 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798"} err="failed to get container status \"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798\": rpc error: code = NotFound desc = could not find container \"b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798\": container with ID starting with b2362353b4b8928658b16962cfd557f2fce459246c0eeb61812db58dbea65798 not found: ID does not exist" Sep 10 17:44:05.533392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533372 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:44:05.533645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533634 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="init-config-reloader" Sep 10 17:44:05.533685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533647 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="init-config-reloader" Sep 10 17:44:05.533685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533658 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="config-reloader" Sep 10 17:44:05.533685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533664 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="config-reloader" Sep 10 17:44:05.533685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533676 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy" Sep 10 17:44:05.533685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533682 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533688 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-metric" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533694 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-metric" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533706 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72611a31-79ad-40dd-a69b-e83b1f12a30b" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533711 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="72611a31-79ad-40dd-a69b-e83b1f12a30b" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533717 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-web" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533723 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-web" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533731 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7835c975-e597-4338-b1e5-8cc5a1d3b279" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533736 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7835c975-e597-4338-b1e5-8cc5a1d3b279" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533742 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533747 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533754 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="alertmanager" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533759 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="alertmanager" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533764 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="prom-label-proxy" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533769 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="prom-label-proxy" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533811 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="prom-label-proxy" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533818 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="32dda579-9a30-4a7b-ae51-3c0b0b2b8a63" containerName="console" Sep 10 17:44:05.533822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533824 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-metric" Sep 10 17:44:05.534343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533831 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="alertmanager" Sep 10 17:44:05.534343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533837 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="72611a31-79ad-40dd-a69b-e83b1f12a30b" containerName="console" Sep 10 17:44:05.534343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533842 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="config-reloader" Sep 10 17:44:05.534343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533847 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy" Sep 10 17:44:05.534343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533852 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7835c975-e597-4338-b1e5-8cc5a1d3b279" containerName="console" Sep 10 17:44:05.534343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.533856 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" containerName="kube-rbac-proxy-web" Sep 10 17:44:05.538871 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.538855 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.541713 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.541690 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n79v8\"" Sep 10 17:44:05.541813 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.541777 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 17:44:05.541813 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.541692 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 17:44:05.541813 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.541783 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 17:44:05.541972 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.541869 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 17:44:05.542056 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.542040 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 17:44:05.542114 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.542056 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 17:44:05.542114 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.542069 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 17:44:05.542506 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.542492 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 17:44:05.547557 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.547530 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 17:44:05.554166 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.552683 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:44:05.621809 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.621778 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a35953-20a7-42ca-9095-b89c17510a94" path="/var/lib/kubelet/pods/a4a35953-20a7-42ca-9095-b89c17510a94/volumes" Sep 10 17:44:05.706435 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706413 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706467 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2617bc1-9b0a-41fe-9709-25680cfac182-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706486 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6c4h\" (UniqueName: \"kubernetes.io/projected/b2617bc1-9b0a-41fe-9709-25680cfac182-kube-api-access-c6c4h\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706699 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706594 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2617bc1-9b0a-41fe-9709-25680cfac182-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706699 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706699 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706754 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2617bc1-9b0a-41fe-9709-25680cfac182-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706814 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706800 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-web-config\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706834 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2617bc1-9b0a-41fe-9709-25680cfac182-config-out\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.706957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.706852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2617bc1-9b0a-41fe-9709-25680cfac182-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.807923 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.807861 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.807923 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.807892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.807923 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.807915 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.807941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2617bc1-9b0a-41fe-9709-25680cfac182-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-web-config\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808147 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2617bc1-9b0a-41fe-9709-25680cfac182-config-out\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2617bc1-9b0a-41fe-9709-25680cfac182-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808430 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808430 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808403 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808540 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2617bc1-9b0a-41fe-9709-25680cfac182-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808540 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808468 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6c4h\" (UniqueName: \"kubernetes.io/projected/b2617bc1-9b0a-41fe-9709-25680cfac182-kube-api-access-c6c4h\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.808540 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.808510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2617bc1-9b0a-41fe-9709-25680cfac182-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.809414 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.809034 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2617bc1-9b0a-41fe-9709-25680cfac182-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.809414 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.809042 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2617bc1-9b0a-41fe-9709-25680cfac182-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.809589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.809497 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2617bc1-9b0a-41fe-9709-25680cfac182-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811242 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811161 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811470 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-web-config\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811497 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2617bc1-9b0a-41fe-9709-25680cfac182-config-out\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811671 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2617bc1-9b0a-41fe-9709-25680cfac182-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811731 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811654 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.811966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.811910 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.812664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.812640 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.812760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.812745 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2617bc1-9b0a-41fe-9709-25680cfac182-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.817465 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.817436 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6c4h\" (UniqueName: \"kubernetes.io/projected/b2617bc1-9b0a-41fe-9709-25680cfac182-kube-api-access-c6c4h\") pod \"alertmanager-main-0\" (UID: \"b2617bc1-9b0a-41fe-9709-25680cfac182\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:05.847274 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:05.847257 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 17:44:06.126717 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:06.126694 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 17:44:06.128673 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:44:06.128646 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2617bc1_9b0a_41fe_9709_25680cfac182.slice/crio-d5c0d8af377a5a1245bde7123180593cc408155452136367f25be9e5bd48a421 WatchSource:0}: Error finding container d5c0d8af377a5a1245bde7123180593cc408155452136367f25be9e5bd48a421: Status 404 returned error can't find the container with id d5c0d8af377a5a1245bde7123180593cc408155452136367f25be9e5bd48a421 Sep 10 17:44:06.479336 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:06.479309 2565 generic.go:356] "Generic (PLEG): container finished" podID="b2617bc1-9b0a-41fe-9709-25680cfac182" containerID="e14af73ba82784425764cb2849da032faeb21bb7eac88b99a7cd30482ca75d03" exitCode=0 Sep 10 17:44:06.479720 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:06.479396 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerDied","Data":"e14af73ba82784425764cb2849da032faeb21bb7eac88b99a7cd30482ca75d03"} Sep 10 17:44:06.479720 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:06.479477 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"d5c0d8af377a5a1245bde7123180593cc408155452136367f25be9e5bd48a421"} Sep 10 17:44:07.487213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.487184 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"0c470bc072d2317d6851655857e35d73021aefae2963f00bf2c7f337d914584a"} Sep 10 17:44:07.487550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.487274 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"15d9dc423971b8e44fff4ce61148af147e96ae8502d799f4c93e82961a3a53e7"} Sep 10 17:44:07.487550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.487301 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"c35e2b2e26f360bc6390909930e3abffe3a79d3ebda8064e3e976ab115842866"} Sep 10 17:44:07.487550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.487325 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"b9675e91aacd31d4be9d8e9ecc60f00acf57b389b429e0f807a2a995f06af87d"} Sep 10 17:44:07.487550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.487355 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"359265cae6c888aa970030a5a5b376d07b52399ef73743d31db36b4162781c28"} Sep 10 17:44:07.487550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.487378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2617bc1-9b0a-41fe-9709-25680cfac182","Type":"ContainerStarted","Data":"aab5a8609108ff6d3ef426f7193745b4e129119d7b80b26a6889479de1a1f933"} Sep 10 17:44:07.520588 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:07.520538 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.503751604 podStartE2EDuration="2.520523263s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.771659ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:44:07.518669409 +0000 UTC m=+176.400394550" watchObservedRunningTime="2026-09-10 17:44:07.520523263 +0000 UTC m=+176.402248404" Sep 10 17:44:10.737241 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.737196 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fb7586c4f-qw89q"] Sep 10 17:44:10.740928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.740904 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.754277 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.754249 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb7586c4f-qw89q"] Sep 10 17:44:10.790587 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.790558 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:10.845730 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845704 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-service-ca\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.845822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-oauth-config\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.845822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-config\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.845822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845798 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-trusted-ca-bundle\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.845822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845816 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-oauth-serving-cert\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.846010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-serving-cert\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.846010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.845904 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78mlm\" (UniqueName: \"kubernetes.io/projected/01b8331b-b869-488d-8dcc-2bc4e749dd39-kube-api-access-78mlm\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.886400 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.886377 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:10.946871 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78mlm\" (UniqueName: \"kubernetes.io/projected/01b8331b-b869-488d-8dcc-2bc4e749dd39-kube-api-access-78mlm\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.946936 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-service-ca\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.946936 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946913 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-oauth-config\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.946936 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-config\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947054 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-trusted-ca-bundle\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947054 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-oauth-serving-cert\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947054 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.946989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-serving-cert\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.947633 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-service-ca\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.947686 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-config\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.947691 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-oauth-serving-cert\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.947879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.947796 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-trusted-ca-bundle\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.949478 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.949451 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-oauth-config\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.949575 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.949515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-serving-cert\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:10.954792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:10.954756 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78mlm\" (UniqueName: \"kubernetes.io/projected/01b8331b-b869-488d-8dcc-2bc4e749dd39-kube-api-access-78mlm\") pod \"console-fb7586c4f-qw89q\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:11.019178 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.019129 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:11.051597 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.051580 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:11.072169 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.072135 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:11.324565 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.324541 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb7586c4f-qw89q"] Sep 10 17:44:11.326771 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:44:11.326743 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b8331b_b869_488d_8dcc_2bc4e749dd39.slice/crio-f8850c2ec144614ea40d688b857ae00fba158ad05febc6a609bce84f815dc465 WatchSource:0}: Error finding container f8850c2ec144614ea40d688b857ae00fba158ad05febc6a609bce84f815dc465: Status 404 returned error can't find the container with id f8850c2ec144614ea40d688b857ae00fba158ad05febc6a609bce84f815dc465 Sep 10 17:44:11.504602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.504562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb7586c4f-qw89q" event={"ID":"01b8331b-b869-488d-8dcc-2bc4e749dd39","Type":"ContainerStarted","Data":"a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec"} Sep 10 17:44:11.504804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.504624 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb7586c4f-qw89q" event={"ID":"01b8331b-b869-488d-8dcc-2bc4e749dd39","Type":"ContainerStarted","Data":"f8850c2ec144614ea40d688b857ae00fba158ad05febc6a609bce84f815dc465"} Sep 10 17:44:11.524133 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:11.524091 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fb7586c4f-qw89q" podStartSLOduration=1.524077404 podStartE2EDuration="1.524077404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:44:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:44:11.523345178 +0000 UTC m=+180.405070320" watchObservedRunningTime="2026-09-10 17:44:11.524077404 +0000 UTC m=+180.405802544" Sep 10 17:44:14.143389 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:14.143348 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:14.239666 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:14.239637 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:14.288246 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:14.288202 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:21.052198 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.052149 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:21.052859 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.052240 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:21.057161 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.057137 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:21.548712 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.548688 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:44:21.600253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.600184 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-db77d758-w2g4f"] Sep 10 17:44:21.703008 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.702975 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:21.751491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.751461 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:21.803660 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:21.803586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:46.621704 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.621645 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-db77d758-w2g4f" podUID="90b4d527-70b7-41a8-aceb-e069264422f2" containerName="console" containerID="cri-o://b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3" gracePeriod=15 Sep 10 17:44:46.861709 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.861686 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:44:46.917979 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.917920 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-service-ca\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.917979 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.917956 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k8tr\" (UniqueName: \"kubernetes.io/projected/90b4d527-70b7-41a8-aceb-e069264422f2-kube-api-access-5k8tr\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.917979 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.917973 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-oauth-serving-cert\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.918167 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918104 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-oauth-config\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.918167 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918153 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-serving-cert\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.918274 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918204 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-console-config\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.918274 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918265 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-trusted-ca-bundle\") pod \"90b4d527-70b7-41a8-aceb-e069264422f2\" (UID: \"90b4d527-70b7-41a8-aceb-e069264422f2\") " Sep 10 17:44:46.918387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918300 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:44:46.918443 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918412 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:44:46.918558 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918513 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:46.918558 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918534 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-oauth-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:46.918686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918648 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-console-config" (OuterVolumeSpecName: "console-config") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:44:46.918802 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.918774 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:44:46.920185 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.920160 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:46.920527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.920497 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:44:46.920624 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:46.920591 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b4d527-70b7-41a8-aceb-e069264422f2-kube-api-access-5k8tr" (OuterVolumeSpecName: "kube-api-access-5k8tr") pod "90b4d527-70b7-41a8-aceb-e069264422f2" (UID: "90b4d527-70b7-41a8-aceb-e069264422f2"). InnerVolumeSpecName "kube-api-access-5k8tr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:44:47.019404 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.019380 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-oauth-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:47.019508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.019405 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4d527-70b7-41a8-aceb-e069264422f2-console-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:47.019508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.019418 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-console-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:47.019508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.019426 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4d527-70b7-41a8-aceb-e069264422f2-trusted-ca-bundle\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:47.019508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.019435 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k8tr\" (UniqueName: \"kubernetes.io/projected/90b4d527-70b7-41a8-aceb-e069264422f2-kube-api-access-5k8tr\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:44:47.648200 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.648168 2565 generic.go:356] "Generic (PLEG): container finished" podID="90b4d527-70b7-41a8-aceb-e069264422f2" containerID="b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3" exitCode=0 Sep 10 17:44:47.648615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.648215 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db77d758-w2g4f" Sep 10 17:44:47.648615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.648256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db77d758-w2g4f" event={"ID":"90b4d527-70b7-41a8-aceb-e069264422f2","Type":"ContainerDied","Data":"b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3"} Sep 10 17:44:47.648615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.648305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db77d758-w2g4f" event={"ID":"90b4d527-70b7-41a8-aceb-e069264422f2","Type":"ContainerDied","Data":"beee7306ed7910635baa46f54da48f55eb7cb82751ba2a8fbab91e4bb6a52078"} Sep 10 17:44:47.648615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.648330 2565 scope.go:117] "RemoveContainer" containerID="b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3" Sep 10 17:44:47.656659 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.656634 2565 scope.go:117] "RemoveContainer" containerID="b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3" Sep 10 17:44:47.656905 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:44:47.656887 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3\": container with ID starting with b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3 not found: ID does not exist" containerID="b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3" Sep 10 17:44:47.656954 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.656911 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3"} err="failed to get container status \"b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3\": rpc error: code = NotFound desc = could not find container \"b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3\": container with ID starting with b26dbfd130c2fac5b37436a61d3ced54b4d9b545efd7e64a41e81053e28f7cf3 not found: ID does not exist" Sep 10 17:44:47.663986 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.663957 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-db77d758-w2g4f"] Sep 10 17:44:47.666366 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:47.666344 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-db77d758-w2g4f"] Sep 10 17:44:49.621131 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:49.621095 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b4d527-70b7-41a8-aceb-e069264422f2" path="/var/lib/kubelet/pods/90b4d527-70b7-41a8-aceb-e069264422f2/volumes" Sep 10 17:44:50.169307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:50.169274 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:50.233307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:50.233279 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:52.705413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:44:52.705360 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:06.337317 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:06.337278 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:06.436545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:06.436516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:31.900678 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:31.900645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:32.739269 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:32.739236 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:34.110582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:34.110509 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:36.547287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:36.547256 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:36.607444 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:36.607414 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:38.345698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:38.345659 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:38.445083 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:38.445050 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:39.170622 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.170588 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl"] Sep 10 17:45:39.170897 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.170886 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b4d527-70b7-41a8-aceb-e069264422f2" containerName="console" Sep 10 17:45:39.170939 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.170899 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b4d527-70b7-41a8-aceb-e069264422f2" containerName="console" Sep 10 17:45:39.171001 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.170991 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b4d527-70b7-41a8-aceb-e069264422f2" containerName="console" Sep 10 17:45:39.173974 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.173958 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.179387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.179365 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 17:45:39.179387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.179383 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 17:45:39.180666 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.180650 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s2ppn\"" Sep 10 17:45:39.187116 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.187091 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl"] Sep 10 17:45:39.292988 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.292952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.293176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.293036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pqnk\" (UniqueName: \"kubernetes.io/projected/24659d13-a727-4493-b7c7-df0828c36553-kube-api-access-9pqnk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.293176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.293096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.394325 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.394275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pqnk\" (UniqueName: \"kubernetes.io/projected/24659d13-a727-4493-b7c7-df0828c36553-kube-api-access-9pqnk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.394804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.394353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.394804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.394393 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.394804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.394718 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.394937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.394812 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.404260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.404236 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pqnk\" (UniqueName: \"kubernetes.io/projected/24659d13-a727-4493-b7c7-df0828c36553-kube-api-access-9pqnk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.482362 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.482292 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:39.751551 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.751524 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl"] Sep 10 17:45:39.753854 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:45:39.753827 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24659d13_a727_4493_b7c7_df0828c36553.slice/crio-87b0fa02e999059127fcfa9c6481747f20de87cc9e814d7b0fa071afec048a8b WatchSource:0}: Error finding container 87b0fa02e999059127fcfa9c6481747f20de87cc9e814d7b0fa071afec048a8b: Status 404 returned error can't find the container with id 87b0fa02e999059127fcfa9c6481747f20de87cc9e814d7b0fa071afec048a8b Sep 10 17:45:39.844149 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:39.844109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" event={"ID":"24659d13-a727-4493-b7c7-df0828c36553","Type":"ContainerStarted","Data":"87b0fa02e999059127fcfa9c6481747f20de87cc9e814d7b0fa071afec048a8b"} Sep 10 17:45:40.826797 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:40.826764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:41.086085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:41.085993 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:41.709043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:41.709002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:41.761080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:41.761048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:43.348693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:43.348650 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:43.449246 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:43.449196 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:45.600541 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.600505 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fmpjh"] Sep 10 17:45:45.605112 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.605087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.607716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.607690 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 17:45:45.608848 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.608814 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 17:45:45.608948 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.608815 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-44vpd\"" Sep 10 17:45:45.614530 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.614480 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fmpjh"] Sep 10 17:45:45.647357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.647307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c100f9b0-4e02-46dc-b9a1-565f8023677e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fmpjh\" (UID: \"c100f9b0-4e02-46dc-b9a1-565f8023677e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.647538 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.647408 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nsnf\" (UniqueName: \"kubernetes.io/projected/c100f9b0-4e02-46dc-b9a1-565f8023677e-kube-api-access-5nsnf\") pod \"cert-manager-webhook-6486ffd997-fmpjh\" (UID: \"c100f9b0-4e02-46dc-b9a1-565f8023677e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.748211 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.748188 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c100f9b0-4e02-46dc-b9a1-565f8023677e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fmpjh\" (UID: \"c100f9b0-4e02-46dc-b9a1-565f8023677e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.748324 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.748245 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nsnf\" (UniqueName: \"kubernetes.io/projected/c100f9b0-4e02-46dc-b9a1-565f8023677e-kube-api-access-5nsnf\") pod \"cert-manager-webhook-6486ffd997-fmpjh\" (UID: \"c100f9b0-4e02-46dc-b9a1-565f8023677e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.755792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.755767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c100f9b0-4e02-46dc-b9a1-565f8023677e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fmpjh\" (UID: \"c100f9b0-4e02-46dc-b9a1-565f8023677e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.755908 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.755892 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nsnf\" (UniqueName: \"kubernetes.io/projected/c100f9b0-4e02-46dc-b9a1-565f8023677e-kube-api-access-5nsnf\") pod \"cert-manager-webhook-6486ffd997-fmpjh\" (UID: \"c100f9b0-4e02-46dc-b9a1-565f8023677e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:45.872595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.872530 2565 generic.go:356] "Generic (PLEG): container finished" podID="24659d13-a727-4493-b7c7-df0828c36553" containerID="d371a9b210e6d5b5be0ba70d5fb60a0c720f31d0e8b58d6147d00a04bbc06893" exitCode=0 Sep 10 17:45:45.872595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.872571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" event={"ID":"24659d13-a727-4493-b7c7-df0828c36553","Type":"ContainerDied","Data":"d371a9b210e6d5b5be0ba70d5fb60a0c720f31d0e8b58d6147d00a04bbc06893"} Sep 10 17:45:45.926678 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:45.926651 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:46.197241 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:46.197195 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fmpjh"] Sep 10 17:45:46.199139 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:45:46.199104 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc100f9b0_4e02_46dc_b9a1_565f8023677e.slice/crio-69e44f90190762dfea7b8d6de4d12c5f1b6625d1cb0dade4966a16676390e37f WatchSource:0}: Error finding container 69e44f90190762dfea7b8d6de4d12c5f1b6625d1cb0dade4966a16676390e37f: Status 404 returned error can't find the container with id 69e44f90190762dfea7b8d6de4d12c5f1b6625d1cb0dade4966a16676390e37f Sep 10 17:45:46.880615 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:46.880574 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" event={"ID":"c100f9b0-4e02-46dc-b9a1-565f8023677e","Type":"ContainerStarted","Data":"69e44f90190762dfea7b8d6de4d12c5f1b6625d1cb0dade4966a16676390e37f"} Sep 10 17:45:47.837359 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:47.837316 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:48.643590 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.643558 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b69sm"] Sep 10 17:45:48.647205 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.647180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.650070 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.650044 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-md7sh\"" Sep 10 17:45:48.658582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.658560 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b69sm"] Sep 10 17:45:48.671957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.671925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ad9de9e-a5f1-4417-9a8a-3b14226f17c6-bound-sa-token\") pod \"cert-manager-55b4d954d5-b69sm\" (UID: \"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6\") " pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.672066 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.672000 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pgvj\" (UniqueName: \"kubernetes.io/projected/0ad9de9e-a5f1-4417-9a8a-3b14226f17c6-kube-api-access-5pgvj\") pod \"cert-manager-55b4d954d5-b69sm\" (UID: \"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6\") " pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.772746 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.772707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ad9de9e-a5f1-4417-9a8a-3b14226f17c6-bound-sa-token\") pod \"cert-manager-55b4d954d5-b69sm\" (UID: \"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6\") " pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.772924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.772800 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pgvj\" (UniqueName: \"kubernetes.io/projected/0ad9de9e-a5f1-4417-9a8a-3b14226f17c6-kube-api-access-5pgvj\") pod \"cert-manager-55b4d954d5-b69sm\" (UID: \"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6\") " pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.781844 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.781819 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ad9de9e-a5f1-4417-9a8a-3b14226f17c6-bound-sa-token\") pod \"cert-manager-55b4d954d5-b69sm\" (UID: \"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6\") " pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.782029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.782011 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pgvj\" (UniqueName: \"kubernetes.io/projected/0ad9de9e-a5f1-4417-9a8a-3b14226f17c6-kube-api-access-5pgvj\") pod \"cert-manager-55b4d954d5-b69sm\" (UID: \"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6\") " pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:48.957781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:48.957753 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b69sm" Sep 10 17:45:50.226291 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.226267 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b69sm"] Sep 10 17:45:50.228441 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:45:50.228415 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ad9de9e_a5f1_4417_9a8a_3b14226f17c6.slice/crio-fa68ae5d62d362b0a9e991695b8961dfc59f18c563c176749f31a97530dfa1b1 WatchSource:0}: Error finding container fa68ae5d62d362b0a9e991695b8961dfc59f18c563c176749f31a97530dfa1b1: Status 404 returned error can't find the container with id fa68ae5d62d362b0a9e991695b8961dfc59f18c563c176749f31a97530dfa1b1 Sep 10 17:45:50.899838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.899801 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" event={"ID":"c100f9b0-4e02-46dc-b9a1-565f8023677e","Type":"ContainerStarted","Data":"4406d11edb96d9b6f9c5bf7414552a818a4fb9b8bbe4780a88062814ade5916b"} Sep 10 17:45:50.900032 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.899879 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:50.901329 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.901299 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b69sm" event={"ID":"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6","Type":"ContainerStarted","Data":"45d02f43bfe1bdf6442904a9f116001d9696af29697558327ed05a8f7351e5a4"} Sep 10 17:45:50.901443 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.901339 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b69sm" event={"ID":"0ad9de9e-a5f1-4417-9a8a-3b14226f17c6","Type":"ContainerStarted","Data":"fa68ae5d62d362b0a9e991695b8961dfc59f18c563c176749f31a97530dfa1b1"} Sep 10 17:45:50.902979 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.902942 2565 generic.go:356] "Generic (PLEG): container finished" podID="24659d13-a727-4493-b7c7-df0828c36553" containerID="7c033748266d60d37b54d93007e51a64cfac13eccfa4502393bd141ea7d25803" exitCode=0 Sep 10 17:45:50.903109 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.902993 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" event={"ID":"24659d13-a727-4493-b7c7-df0828c36553","Type":"ContainerDied","Data":"7c033748266d60d37b54d93007e51a64cfac13eccfa4502393bd141ea7d25803"} Sep 10 17:45:50.918633 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.918594 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" podStartSLOduration=2.149754743 podStartE2EDuration="5.918579776s" totalImagesPullingTime="3.768825033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:45:50.916478277 +0000 UTC m=+279.798203418" watchObservedRunningTime="2026-09-10 17:45:50.918579776 +0000 UTC m=+279.800304917" Sep 10 17:45:50.932525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:50.932487 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-b69sm" podStartSLOduration=2.932474283 podStartE2EDuration="2.932474283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:45:50.931141749 +0000 UTC m=+279.812866890" watchObservedRunningTime="2026-09-10 17:45:50.932474283 +0000 UTC m=+279.814199424" Sep 10 17:45:51.824642 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:51.824606 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:51.882055 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:51.882025 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:56.912131 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:56.912096 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fmpjh" Sep 10 17:45:57.938852 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:57.938815 2565 generic.go:356] "Generic (PLEG): container finished" podID="24659d13-a727-4493-b7c7-df0828c36553" containerID="4d519dba7b2cc38a80916e9373e88446d8e3bb70c248421656ca8fad94b42266" exitCode=0 Sep 10 17:45:57.939198 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:57.938868 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" event={"ID":"24659d13-a727-4493-b7c7-df0828c36553","Type":"ContainerDied","Data":"4d519dba7b2cc38a80916e9373e88446d8e3bb70c248421656ca8fad94b42266"} Sep 10 17:45:59.111996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.111962 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:59.156865 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.156839 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pqnk\" (UniqueName: \"kubernetes.io/projected/24659d13-a727-4493-b7c7-df0828c36553-kube-api-access-9pqnk\") pod \"24659d13-a727-4493-b7c7-df0828c36553\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " Sep 10 17:45:59.157023 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.156870 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-util\") pod \"24659d13-a727-4493-b7c7-df0828c36553\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " Sep 10 17:45:59.157023 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.156895 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-bundle\") pod \"24659d13-a727-4493-b7c7-df0828c36553\" (UID: \"24659d13-a727-4493-b7c7-df0828c36553\") " Sep 10 17:45:59.157328 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.157302 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-bundle" (OuterVolumeSpecName: "bundle") pod "24659d13-a727-4493-b7c7-df0828c36553" (UID: "24659d13-a727-4493-b7c7-df0828c36553"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:45:59.158966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.158941 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24659d13-a727-4493-b7c7-df0828c36553-kube-api-access-9pqnk" (OuterVolumeSpecName: "kube-api-access-9pqnk") pod "24659d13-a727-4493-b7c7-df0828c36553" (UID: "24659d13-a727-4493-b7c7-df0828c36553"). InnerVolumeSpecName "kube-api-access-9pqnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:45:59.161319 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.161296 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-util" (OuterVolumeSpecName: "util") pod "24659d13-a727-4493-b7c7-df0828c36553" (UID: "24659d13-a727-4493-b7c7-df0828c36553"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:45:59.257486 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.257414 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pqnk\" (UniqueName: \"kubernetes.io/projected/24659d13-a727-4493-b7c7-df0828c36553-kube-api-access-9pqnk\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:45:59.257486 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.257443 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-util\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:45:59.257486 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.257457 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24659d13-a727-4493-b7c7-df0828c36553-bundle\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:45:59.949213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.949182 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" Sep 10 17:45:59.949371 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.949212 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f278tl" event={"ID":"24659d13-a727-4493-b7c7-df0828c36553","Type":"ContainerDied","Data":"87b0fa02e999059127fcfa9c6481747f20de87cc9e814d7b0fa071afec048a8b"} Sep 10 17:45:59.949371 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:45:59.949266 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87b0fa02e999059127fcfa9c6481747f20de87cc9e814d7b0fa071afec048a8b" Sep 10 17:46:03.712657 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:03.712622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:03.809930 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:03.809898 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:04.409695 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:04.409658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:04.507045 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:04.507014 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:09.902493 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:09.902462 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:11.108445 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:11.108411 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:11.514774 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:11.514747 2565 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 17:46:11.908514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:11.908435 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:22.179096 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.179059 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:22.260625 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.260586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:22.274456 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274428 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx"] Sep 10 17:46:22.274727 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274716 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="extract" Sep 10 17:46:22.275002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274729 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="extract" Sep 10 17:46:22.275002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274746 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="util" Sep 10 17:46:22.275002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274752 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="util" Sep 10 17:46:22.275002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274759 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="pull" Sep 10 17:46:22.275002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274765 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="pull" Sep 10 17:46:22.275002 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.274822 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="24659d13-a727-4493-b7c7-df0828c36553" containerName="extract" Sep 10 17:46:22.276695 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.276676 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.279485 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.279464 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 17:46:22.279590 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.279501 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 17:46:22.280757 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.280735 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 17:46:22.280877 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.280736 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-t5xjq\"" Sep 10 17:46:22.280877 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.280777 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:46:22.281457 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.281442 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 17:46:22.288029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.288008 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx"] Sep 10 17:46:22.338760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.338736 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-manager-config\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.338852 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.338762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-cert\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.338852 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.338796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zpdb\" (UniqueName: \"kubernetes.io/projected/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-kube-api-access-9zpdb\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.338852 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.338835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-metrics-cert\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.439148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.439093 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-manager-config\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.439148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.439121 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-cert\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.439315 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.439164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zpdb\" (UniqueName: \"kubernetes.io/projected/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-kube-api-access-9zpdb\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.439315 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.439192 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-metrics-cert\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.439830 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.439809 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-manager-config\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.441643 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.441623 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-metrics-cert\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.441755 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.441643 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-cert\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.448302 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.448280 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zpdb\" (UniqueName: \"kubernetes.io/projected/ffc090a4-ca7d-4ded-b01e-017e6f81cad4-kube-api-access-9zpdb\") pod \"lws-controller-manager-746f98bbcc-kn9qx\" (UID: \"ffc090a4-ca7d-4ded-b01e-017e6f81cad4\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.586294 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.586268 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:22.859900 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.859827 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx"] Sep 10 17:46:22.862789 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:46:22.862760 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc090a4_ca7d_4ded_b01e_017e6f81cad4.slice/crio-3260976dbe0b68c295b1d67c2224c7f076ca132aebf0d18cfb8884784571b01b WatchSource:0}: Error finding container 3260976dbe0b68c295b1d67c2224c7f076ca132aebf0d18cfb8884784571b01b: Status 404 returned error can't find the container with id 3260976dbe0b68c295b1d67c2224c7f076ca132aebf0d18cfb8884784571b01b Sep 10 17:46:22.864764 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:22.864747 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:46:23.042148 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:23.042120 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" event={"ID":"ffc090a4-ca7d-4ded-b01e-017e6f81cad4","Type":"ContainerStarted","Data":"3260976dbe0b68c295b1d67c2224c7f076ca132aebf0d18cfb8884784571b01b"} Sep 10 17:46:24.791735 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:24.791698 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:24.882059 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:24.882032 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:25.051461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.051397 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" event={"ID":"ffc090a4-ca7d-4ded-b01e-017e6f81cad4","Type":"ContainerStarted","Data":"237d1d4d0745fc4a33c15e2232db2a837f46ac274ecb7b07dff814c24d0564ed"} Sep 10 17:46:25.051576 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.051494 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:25.069352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.069264 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" podStartSLOduration=1.52860809 podStartE2EDuration="3.069246678s" totalImagesPullingTime="1.540638588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:25.068875809 +0000 UTC m=+313.950600953" watchObservedRunningTime="2026-09-10 17:46:25.069246678 +0000 UTC m=+313.950971817" Sep 10 17:46:25.409119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.409049 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:25.655589 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.655539 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:25.753812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.753786 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:25.843125 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.843090 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99"] Sep 10 17:46:25.864850 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.864830 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:25.872077 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.872055 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 17:46:25.872467 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.872443 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 17:46:25.873391 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.873369 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kbvxn\"" Sep 10 17:46:25.873391 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.873384 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 17:46:25.873662 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.873643 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 17:46:25.874497 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.874478 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99"] Sep 10 17:46:25.964728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.964698 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:25.970640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.970621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1330a45b-d686-4705-9035-8f9bba0f01ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:25.970710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.970657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbj6v\" (UniqueName: \"kubernetes.io/projected/1330a45b-d686-4705-9035-8f9bba0f01ba-kube-api-access-lbj6v\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:25.970710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.970680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1330a45b-d686-4705-9035-8f9bba0f01ba-tmp\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:25.970788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.970742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1330a45b-d686-4705-9035-8f9bba0f01ba-manifests\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:25.970788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:25.970772 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1330a45b-d686-4705-9035-8f9bba0f01ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.071634 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.071584 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1330a45b-d686-4705-9035-8f9bba0f01ba-manifests\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.071634 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.071619 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1330a45b-d686-4705-9035-8f9bba0f01ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.071756 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.071661 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1330a45b-d686-4705-9035-8f9bba0f01ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.071756 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.071687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbj6v\" (UniqueName: \"kubernetes.io/projected/1330a45b-d686-4705-9035-8f9bba0f01ba-kube-api-access-lbj6v\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.071756 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.071711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1330a45b-d686-4705-9035-8f9bba0f01ba-tmp\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.071995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.071973 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1330a45b-d686-4705-9035-8f9bba0f01ba-manifests\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.074093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.074069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1330a45b-d686-4705-9035-8f9bba0f01ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.074188 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.074074 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1330a45b-d686-4705-9035-8f9bba0f01ba-tmp\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.074188 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.074146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1330a45b-d686-4705-9035-8f9bba0f01ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.083861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.083839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:26.105652 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.105629 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbj6v\" (UniqueName: \"kubernetes.io/projected/1330a45b-d686-4705-9035-8f9bba0f01ba-kube-api-access-lbj6v\") pod \"opendatahub-operator-controller-manager-556447cf57-xkr99\" (UID: \"1330a45b-d686-4705-9035-8f9bba0f01ba\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.172930 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.172911 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:26.175867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.175852 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:26.278143 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.278114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:26.368171 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.368110 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:26.434215 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.434185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:26.474685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.474664 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99"] Sep 10 17:46:26.477047 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:46:26.477024 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1330a45b_d686_4705_9035_8f9bba0f01ba.slice/crio-5041cf07408ee7c1ff5a3ccdf7f7f4a14a1ca5b01e4de41e51fd0f2a5c7d6bcc WatchSource:0}: Error finding container 5041cf07408ee7c1ff5a3ccdf7f7f4a14a1ca5b01e4de41e51fd0f2a5c7d6bcc: Status 404 returned error can't find the container with id 5041cf07408ee7c1ff5a3ccdf7f7f4a14a1ca5b01e4de41e51fd0f2a5c7d6bcc Sep 10 17:46:26.495413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.495387 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:26.592603 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:26.592579 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:27.064209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:27.064169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" event={"ID":"1330a45b-d686-4705-9035-8f9bba0f01ba","Type":"ContainerStarted","Data":"5041cf07408ee7c1ff5a3ccdf7f7f4a14a1ca5b01e4de41e51fd0f2a5c7d6bcc"} Sep 10 17:46:30.080852 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:30.080817 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" event={"ID":"1330a45b-d686-4705-9035-8f9bba0f01ba","Type":"ContainerStarted","Data":"eafdb355c9c558108f9b6a5ebf1dd15b657a7965288819b7b69bc9494724e7bf"} Sep 10 17:46:31.086485 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:31.086452 2565 generic.go:356] "Generic (PLEG): container finished" podID="1330a45b-d686-4705-9035-8f9bba0f01ba" containerID="eafdb355c9c558108f9b6a5ebf1dd15b657a7965288819b7b69bc9494724e7bf" exitCode=0 Sep 10 17:46:31.086835 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:31.086511 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" event={"ID":"1330a45b-d686-4705-9035-8f9bba0f01ba","Type":"ContainerDied","Data":"eafdb355c9c558108f9b6a5ebf1dd15b657a7965288819b7b69bc9494724e7bf"} Sep 10 17:46:32.093043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:32.093002 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" event={"ID":"1330a45b-d686-4705-9035-8f9bba0f01ba","Type":"ContainerStarted","Data":"7823b04a3c496bbd1a4e19703bc79dfc2a759b8b3c04de21c8ee15c545245a62"} Sep 10 17:46:32.093535 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:32.093143 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:32.117574 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:32.117523 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" podStartSLOduration=2.609165214 podStartE2EDuration="7.117507299s" totalImagesPullingTime="3.501464973s" totalInitContainerRuntime="1.006877112s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:32.116450448 +0000 UTC m=+320.998175590" watchObservedRunningTime="2026-09-10 17:46:32.117507299 +0000 UTC m=+320.999232440" Sep 10 17:46:36.057591 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:36.057553 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-kn9qx" Sep 10 17:46:37.724665 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:37.724634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:37.828514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:37.828484 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:43.099680 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:43.099652 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-xkr99" Sep 10 17:46:45.268677 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:45.268644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:45.395045 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:45.395002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:45.698159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:45.698114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:45.934030 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:45.933995 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.303119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:47.303090 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.400561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:47.400526 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.474080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:47.474048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:48.706460 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:48.706426 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:52.525108 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.525062 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:52.606793 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.606763 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:52.805831 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.805762 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5"] Sep 10 17:46:52.812472 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.812453 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.841855 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.841835 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8kvsp\"" Sep 10 17:46:52.842415 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.842401 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 17:46:52.865356 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.865332 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5"] Sep 10 17:46:52.870508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.870487 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:52.874573 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874554 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874694 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874749 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874723 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htwkl\" (UniqueName: \"kubernetes.io/projected/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-kube-api-access-htwkl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874763 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874852 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874886 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874934 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874887 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.874934 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.874912 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.975716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.975879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.975879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.975879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975819 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.975879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975847 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976102 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975885 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976102 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htwkl\" (UniqueName: \"kubernetes.io/projected/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-kube-api-access-htwkl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976102 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975958 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976102 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.975996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976335 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.976310 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976394 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.976340 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976394 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.976384 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.976547 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.976569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.976560 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.978016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.977996 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:52.978296 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:52.978277 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:53.013334 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:53.013310 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htwkl\" (UniqueName: \"kubernetes.io/projected/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-kube-api-access-htwkl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:53.013421 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:53.013372 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5\" (UID: \"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:53.098169 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:53.098112 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:53.122706 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:53.122688 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:53.199450 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:53.199420 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:53.402996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:53.402923 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5"] Sep 10 17:46:53.406595 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:46:53.406566 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2696ef9_6d15_4bab_9e50_dd1b4b9a7dce.slice/crio-af7c5b8c41ef3b6194f729da5e277e74e970556394996d8102adc7281cd26a81 WatchSource:0}: Error finding container af7c5b8c41ef3b6194f729da5e277e74e970556394996d8102adc7281cd26a81: Status 404 returned error can't find the container with id af7c5b8c41ef3b6194f729da5e277e74e970556394996d8102adc7281cd26a81 Sep 10 17:46:54.195419 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:54.195378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" event={"ID":"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce","Type":"ContainerStarted","Data":"af7c5b8c41ef3b6194f729da5e277e74e970556394996d8102adc7281cd26a81"} Sep 10 17:46:56.031187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.031148 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:46:56.031488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.031241 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:46:56.031488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.031270 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:46:56.207363 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.207326 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" event={"ID":"e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce","Type":"ContainerStarted","Data":"4697c93179bef51a0e1b1d145062a08587dce46e69cbda363e85fc1d4ea2b5bf"} Sep 10 17:46:56.232060 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.232007 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" podStartSLOduration=1.60961815 podStartE2EDuration="4.231991733s" totalImagesPullingTime="2.622373583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:56.230664664 +0000 UTC m=+345.112389806" watchObservedRunningTime="2026-09-10 17:46:56.231991733 +0000 UTC m=+345.113716874" Sep 10 17:46:56.421264 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.421218 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gnb4w"] Sep 10 17:46:56.425177 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.425160 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:56.427829 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.427802 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 10 17:46:56.427954 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.427869 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-djgnx\"" Sep 10 17:46:56.434319 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.434295 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gnb4w"] Sep 10 17:46:56.437147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.437105 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:56.495584 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.495552 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:56.507385 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.507359 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcgzb\" (UniqueName: \"kubernetes.io/projected/c4e74032-3bc2-4023-9c9c-23961ef64df3-kube-api-access-gcgzb\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:56.507497 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.507398 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e74032-3bc2-4023-9c9c-23961ef64df3-cert\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:56.594321 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.594294 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:56.607978 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.607952 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e74032-3bc2-4023-9c9c-23961ef64df3-cert\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:56.608100 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.608020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcgzb\" (UniqueName: \"kubernetes.io/projected/c4e74032-3bc2-4023-9c9c-23961ef64df3-kube-api-access-gcgzb\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:56.608147 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:46:56.608111 2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 10 17:46:56.608191 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:46:56.608177 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e74032-3bc2-4023-9c9c-23961ef64df3-cert podName:c4e74032-3bc2-4023-9c9c-23961ef64df3 nodeName:}" failed. No retries permitted until 2026-09-10 17:46:57.108159725 +0000 UTC m=+345.989884852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c4e74032-3bc2-4023-9c9c-23961ef64df3-cert") pod "odh-model-controller-7998c67dc4-gnb4w" (UID: "c4e74032-3bc2-4023-9c9c-23961ef64df3") : secret "odh-model-controller-webhook-cert" not found Sep 10 17:46:56.616553 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.616529 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcgzb\" (UniqueName: \"kubernetes.io/projected/c4e74032-3bc2-4023-9c9c-23961ef64df3-kube-api-access-gcgzb\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:56.695021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.694957 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:56.927730 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:56.927703 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:57.083099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.083022 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:57.111840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.111812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e74032-3bc2-4023-9c9c-23961ef64df3-cert\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:57.114198 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.114172 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e74032-3bc2-4023-9c9c-23961ef64df3-cert\") pod \"odh-model-controller-7998c67dc4-gnb4w\" (UID: \"c4e74032-3bc2-4023-9c9c-23961ef64df3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:57.123791 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.123763 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:57.128683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.128653 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:57.212488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.212457 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:57.213412 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.213389 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5" Sep 10 17:46:57.336565 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.336466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:46:57.611171 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:57.611146 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gnb4w"] Sep 10 17:46:57.613202 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:46:57.613177 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e74032_3bc2_4023_9c9c_23961ef64df3.slice/crio-3e7bb4482884aee9fbe6442558a6368f4e98d46176b366c81c0fb97b0c1b3423 WatchSource:0}: Error finding container 3e7bb4482884aee9fbe6442558a6368f4e98d46176b366c81c0fb97b0c1b3423: Status 404 returned error can't find the container with id 3e7bb4482884aee9fbe6442558a6368f4e98d46176b366c81c0fb97b0c1b3423 Sep 10 17:46:58.218033 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:46:58.218000 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" event={"ID":"c4e74032-3bc2-4023-9c9c-23961ef64df3","Type":"ContainerStarted","Data":"3e7bb4482884aee9fbe6442558a6368f4e98d46176b366c81c0fb97b0c1b3423"} Sep 10 17:47:01.233864 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:01.233830 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" event={"ID":"c4e74032-3bc2-4023-9c9c-23961ef64df3","Type":"ContainerStarted","Data":"065dd9dbf607c7ea54b5659c17a16383e4c9b71059596442c2f5a6dd6d0ba85e"} Sep 10 17:47:01.234425 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:01.233918 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:47:01.255639 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:01.255594 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" podStartSLOduration=2.633620728 podStartE2EDuration="5.255579586s" totalImagesPullingTime="2.621958858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:01.255056508 +0000 UTC m=+350.136781650" watchObservedRunningTime="2026-09-10 17:47:01.255579586 +0000 UTC m=+350.137304725" Sep 10 17:47:03.916679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.916597 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r"] Sep 10 17:47:03.921439 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.921419 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.925204 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.925181 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 17:47:03.925351 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.925264 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 17:47:03.925351 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.925325 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lgfjq\"" Sep 10 17:47:03.925500 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.925482 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 17:47:03.941253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.941208 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r"] Sep 10 17:47:03.968780 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968745 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.968961 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.968961 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.968961 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.968961 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.968961 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q52s\" (UniqueName: \"kubernetes.io/projected/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-kube-api-access-6q52s\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:03.969205 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:03.968982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.069754 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069715 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.069754 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.070029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.070029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.070029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.070029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q52s\" (UniqueName: \"kubernetes.io/projected/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-kube-api-access-6q52s\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.070029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.069892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.070803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.070767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.072213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.072179 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.072437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.072414 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.072509 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.072471 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.072561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.072512 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.096905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.096871 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.097106 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.097087 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q52s\" (UniqueName: \"kubernetes.io/projected/74e78f6a-ba52-4e5a-bb21-9aaeb1072565-kube-api-access-6q52s\") pod \"istiod-openshift-gateway-5677ccb49b-brd5r\" (UID: \"74e78f6a-ba52-4e5a-bb21-9aaeb1072565\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.230789 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.230738 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:04.516661 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:04.516597 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r"] Sep 10 17:47:04.518663 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:47:04.518637 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e78f6a_ba52_4e5a_bb21_9aaeb1072565.slice/crio-eb9c000644327ad0a1e6c690f1cb9cf113d7f89332704e98bb80e7dae081edbb WatchSource:0}: Error finding container eb9c000644327ad0a1e6c690f1cb9cf113d7f89332704e98bb80e7dae081edbb: Status 404 returned error can't find the container with id eb9c000644327ad0a1e6c690f1cb9cf113d7f89332704e98bb80e7dae081edbb Sep 10 17:47:05.255881 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:05.255837 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" event={"ID":"74e78f6a-ba52-4e5a-bb21-9aaeb1072565","Type":"ContainerStarted","Data":"eb9c000644327ad0a1e6c690f1cb9cf113d7f89332704e98bb80e7dae081edbb"} Sep 10 17:47:06.686704 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:06.686666 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:47:06.686968 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:06.686734 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:47:07.277907 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.277865 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" event={"ID":"74e78f6a-ba52-4e5a-bb21-9aaeb1072565","Type":"ContainerStarted","Data":"bce1a1d51a7e9750d8b96c618d7cd8c26f272f24fda9abf7f634941240073b89"} Sep 10 17:47:07.278156 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.278133 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:07.279510 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.279477 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-brd5r container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 17:47:07.279623 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.279542 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" podUID="74e78f6a-ba52-4e5a-bb21-9aaeb1072565" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:47:07.365679 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.365602 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" podStartSLOduration=2.199812554 podStartE2EDuration="4.365586468s" totalImagesPullingTime="2.165773914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:07.363695652 +0000 UTC m=+356.245420793" watchObservedRunningTime="2026-09-10 17:47:07.365586468 +0000 UTC m=+356.247311608" Sep 10 17:47:07.399296 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.399214 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:07.489735 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.489698 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:07.692508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.692465 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:07.851177 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:07.851135 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:08.282794 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:08.282768 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-brd5r" Sep 10 17:47:08.827119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:08.827079 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:08.941147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:08.941114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:09.011387 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:09.011355 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:12.240808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:12.240776 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gnb4w" Sep 10 17:47:30.241414 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.241379 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7465887497-dcj9v"] Sep 10 17:47:30.251454 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.251431 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.261566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.261538 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7465887497-dcj9v"] Sep 10 17:47:30.306539 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.306506 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:30.361898 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.361872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:30.382378 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382352 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-service-ca\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.382514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57d47\" (UniqueName: \"kubernetes.io/projected/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-kube-api-access-57d47\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.382514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382460 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-oauth-serving-cert\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.382599 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-serving-cert\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.382599 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382562 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-oauth-config\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.382599 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382589 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-config\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.382722 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.382610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-trusted-ca-bundle\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483281 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-service-ca\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57d47\" (UniqueName: \"kubernetes.io/projected/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-kube-api-access-57d47\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-oauth-serving-cert\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-serving-cert\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-oauth-config\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483392 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-config\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.483426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.483416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-trusted-ca-bundle\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.484083 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.484055 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-service-ca\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.484083 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.484080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-config\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.484261 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.484132 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-oauth-serving-cert\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.484312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.484257 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-trusted-ca-bundle\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.485867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.485851 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-serving-cert\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.485930 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.485880 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-console-oauth-config\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.492654 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.492583 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57d47\" (UniqueName: \"kubernetes.io/projected/bfcf0ac4-3ea1-474b-9902-c97a9d15dfca-kube-api-access-57d47\") pod \"console-7465887497-dcj9v\" (UID: \"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca\") " pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.561995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.561969 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:30.835455 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:30.835432 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7465887497-dcj9v"] Sep 10 17:47:30.837699 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:47:30.837663 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfcf0ac4_3ea1_474b_9902_c97a9d15dfca.slice/crio-3fa56a96266ce7760be0c44cb3c89888aa59baa21c07dd9fc716407e161873e3 WatchSource:0}: Error finding container 3fa56a96266ce7760be0c44cb3c89888aa59baa21c07dd9fc716407e161873e3: Status 404 returned error can't find the container with id 3fa56a96266ce7760be0c44cb3c89888aa59baa21c07dd9fc716407e161873e3 Sep 10 17:47:31.389976 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:31.389945 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7465887497-dcj9v" event={"ID":"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca","Type":"ContainerStarted","Data":"48109d2c25c26dacd11061ac8b08a598ecd4a8e2175239a27ab11d16740b5b79"} Sep 10 17:47:31.390355 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:31.389993 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7465887497-dcj9v" event={"ID":"bfcf0ac4-3ea1-474b-9902-c97a9d15dfca","Type":"ContainerStarted","Data":"3fa56a96266ce7760be0c44cb3c89888aa59baa21c07dd9fc716407e161873e3"} Sep 10 17:47:31.408770 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:31.408725 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7465887497-dcj9v" podStartSLOduration=1.408709191 podStartE2EDuration="1.408709191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:31.408267073 +0000 UTC m=+380.289992215" watchObservedRunningTime="2026-09-10 17:47:31.408709191 +0000 UTC m=+380.290434404" Sep 10 17:47:35.538879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:35.538835 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:40.562496 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:40.562462 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:40.562924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:40.562731 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:40.567515 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:40.567495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:41.438659 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:41.438623 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7465887497-dcj9v" Sep 10 17:47:41.492543 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:41.492507 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb7586c4f-qw89q"] Sep 10 17:47:41.595792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:41.595741 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:41.651862 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:41.651834 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:41.704076 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:41.703968 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:57.699903 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:57.699869 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:57.795426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:47:57.795387 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:06.515378 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.515319 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fb7586c4f-qw89q" podUID="01b8331b-b869-488d-8dcc-2bc4e749dd39" containerName="console" containerID="cri-o://a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec" gracePeriod=15 Sep 10 17:48:06.759628 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.759606 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:48:06.868283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868197 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-oauth-serving-cert\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868283 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868263 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-serving-cert\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868317 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-oauth-config\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868341 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78mlm\" (UniqueName: \"kubernetes.io/projected/01b8331b-b869-488d-8dcc-2bc4e749dd39-kube-api-access-78mlm\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868377 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-service-ca\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868445 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-config\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868477 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-trusted-ca-bundle\") pod \"01b8331b-b869-488d-8dcc-2bc4e749dd39\" (UID: \"01b8331b-b869-488d-8dcc-2bc4e749dd39\") " Sep 10 17:48:06.868801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868604 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:48:06.868801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868744 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-service-ca" (OuterVolumeSpecName: "service-ca") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:48:06.868801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868791 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-oauth-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:06.869057 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.868989 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:48:06.869057 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.869006 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-config" (OuterVolumeSpecName: "console-config") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:48:06.870637 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.870609 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b8331b-b869-488d-8dcc-2bc4e749dd39-kube-api-access-78mlm" (OuterVolumeSpecName: "kube-api-access-78mlm") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "kube-api-access-78mlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:48:06.870747 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.870693 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:48:06.870807 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.870767 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "01b8331b-b869-488d-8dcc-2bc4e749dd39" (UID: "01b8331b-b869-488d-8dcc-2bc4e749dd39"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:48:06.969820 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.969798 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:06.969820 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.969819 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-trusted-ca-bundle\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:06.969971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.969829 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-serving-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:06.969971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.969838 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b8331b-b869-488d-8dcc-2bc4e749dd39-console-oauth-config\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:06.969971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.969848 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78mlm\" (UniqueName: \"kubernetes.io/projected/01b8331b-b869-488d-8dcc-2bc4e749dd39-kube-api-access-78mlm\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:06.969971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:06.969857 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8331b-b869-488d-8dcc-2bc4e749dd39-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:48:07.556850 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.556813 2565 generic.go:356] "Generic (PLEG): container finished" podID="01b8331b-b869-488d-8dcc-2bc4e749dd39" containerID="a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec" exitCode=0 Sep 10 17:48:07.557213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.556859 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb7586c4f-qw89q" Sep 10 17:48:07.557213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.556897 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb7586c4f-qw89q" event={"ID":"01b8331b-b869-488d-8dcc-2bc4e749dd39","Type":"ContainerDied","Data":"a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec"} Sep 10 17:48:07.557213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.556944 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb7586c4f-qw89q" event={"ID":"01b8331b-b869-488d-8dcc-2bc4e749dd39","Type":"ContainerDied","Data":"f8850c2ec144614ea40d688b857ae00fba158ad05febc6a609bce84f815dc465"} Sep 10 17:48:07.557213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.556965 2565 scope.go:117] "RemoveContainer" containerID="a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec" Sep 10 17:48:07.566515 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.566499 2565 scope.go:117] "RemoveContainer" containerID="a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec" Sep 10 17:48:07.566770 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:48:07.566753 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec\": container with ID starting with a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec not found: ID does not exist" containerID="a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec" Sep 10 17:48:07.566832 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.566775 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec"} err="failed to get container status \"a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec\": rpc error: code = NotFound desc = could not find container \"a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec\": container with ID starting with a1d1b8b72636ffd8cb6ef6c892b5b3c88769dbe4b7bd56fc4b225582439b11ec not found: ID does not exist" Sep 10 17:48:07.580622 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.580598 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb7586c4f-qw89q"] Sep 10 17:48:07.582266 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.582247 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fb7586c4f-qw89q"] Sep 10 17:48:07.622424 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:07.622401 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b8331b-b869-488d-8dcc-2bc4e749dd39" path="/var/lib/kubelet/pods/01b8331b-b869-488d-8dcc-2bc4e749dd39/volumes" Sep 10 17:48:10.491386 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:10.491352 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:12.182453 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:12.182408 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:13.302024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:13.301986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:14.239865 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:14.239829 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:14.337517 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:14.337475 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:14.889163 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:14.889118 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:15.117317 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:15.117283 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:15.284827 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:15.284793 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:15.384743 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:15.384707 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:16.528715 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:16.528679 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:17.756870 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:17.756842 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:17.881085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:17.881054 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:18.500815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:18.500776 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:18.740021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:18.739992 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:18.843641 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:18.843570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:18.944835 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:18.944807 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:19.049851 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:19.049824 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:24.301655 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:24.301619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:24.399450 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:24.399425 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:26.763009 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:26.762977 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:26.852586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:26.852559 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:29.835806 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:29.835768 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:30.438198 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:30.438167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:32.105964 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:32.105928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:32.205610 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:32.205587 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:36.871091 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:36.871015 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:36.972619 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:36.972583 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:37.057997 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:37.057963 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:43.496322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:43.496290 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:43.603692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:43.603656 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:46.487053 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:46.487013 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:46.589352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:46.589317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:52.126129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:52.126095 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:52.225236 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:52.225199 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:52.709604 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:52.709574 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:52.765152 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:52.765113 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:59.504308 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:59.504277 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:59.609853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:59.609821 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:59.699607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:59.699577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:59.800268 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:48:59.800145 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:00.682207 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:00.682173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:00.782577 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:00.782549 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:10.560124 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:10.560080 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:12.739602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.739566 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7"] Sep 10 17:49:12.740097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.740081 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b8331b-b869-488d-8dcc-2bc4e749dd39" containerName="console" Sep 10 17:49:12.740154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.740099 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b8331b-b869-488d-8dcc-2bc4e749dd39" containerName="console" Sep 10 17:49:12.740197 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.740166 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b8331b-b869-488d-8dcc-2bc4e749dd39" containerName="console" Sep 10 17:49:12.743359 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.743338 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.751494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.751465 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-t6hqx\"" Sep 10 17:49:12.766879 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.766854 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7"] Sep 10 17:49:12.770869 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.770845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.770983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.770877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.770983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.770894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4cnx\" (UniqueName: \"kubernetes.io/projected/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-kube-api-access-c4cnx\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.770983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.770914 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.771150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.771002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.771150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.771080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.771150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.771111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.771150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.771141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.771369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.771216 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.771668 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.771649 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:12.872523 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872523 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872712 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4cnx\" (UniqueName: \"kubernetes.io/projected/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-kube-api-access-c4cnx\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872712 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872781 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872909 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.872981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.872917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.873145 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.873111 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.873263 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.873247 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.873352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.873312 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.873464 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.873442 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.874820 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.874802 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.875058 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.875038 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.883734 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.883707 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:12.884010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:12.883994 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4cnx\" (UniqueName: \"kubernetes.io/projected/ac2ec7ce-2b19-4a7e-ba20-f88f490d804e-kube-api-access-c4cnx\") pod \"maas-default-gateway-openshift-default-5ccb968c66-n7pc7\" (UID: \"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:13.054417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.054346 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:13.160582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.160549 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:13.330548 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.330521 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7"] Sep 10 17:49:13.332776 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:49:13.332749 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2ec7ce_2b19_4a7e_ba20_f88f490d804e.slice/crio-64ac00322b63c44e5e254c558a442bd6c2859a06f290c6008093e01133ca45cb WatchSource:0}: Error finding container 64ac00322b63c44e5e254c558a442bd6c2859a06f290c6008093e01133ca45cb: Status 404 returned error can't find the container with id 64ac00322b63c44e5e254c558a442bd6c2859a06f290c6008093e01133ca45cb Sep 10 17:49:13.334894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.334860 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:49:13.334950 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.334924 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:49:13.334989 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.334952 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:49:13.851966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.851931 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" event={"ID":"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e","Type":"ContainerStarted","Data":"d7cf5494231a071fd9593d5fe238877d0383cde6889beaf18d5dfd40ff367e1c"} Sep 10 17:49:13.852391 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.851990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" event={"ID":"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e","Type":"ContainerStarted","Data":"64ac00322b63c44e5e254c558a442bd6c2859a06f290c6008093e01133ca45cb"} Sep 10 17:49:13.875771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:13.875721 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" podStartSLOduration=1.875703857 podStartE2EDuration="1.875703857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:13.874772505 +0000 UTC m=+482.756497648" watchObservedRunningTime="2026-09-10 17:49:13.875703857 +0000 UTC m=+482.757429006" Sep 10 17:49:14.055092 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:14.055055 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:14.059674 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:14.059650 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:14.856896 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:14.856860 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:14.858273 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:14.858254 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:49:27.771120 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:27.771091 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:27.858373 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:27.858342 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:27.907951 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:27.907901 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:28.633481 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:28.633444 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:28.712969 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:28.712941 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:46.555663 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:46.555619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:46.651718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:46.651668 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:47.081697 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:47.081669 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:56.593734 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:56.593697 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:56.694097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:49:56.694067 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:03.045025 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:03.044991 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:03.139550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:03.139515 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:04.009789 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:04.009712 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:04.109692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:04.109658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:04.161147 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:04.161120 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:04.422288 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:04.422262 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:04.591195 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:04.591164 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:08.095123 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:08.095092 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:08.202281 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:08.202250 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:40.825296 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:40.825239 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:40.922832 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:40.922781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.246098 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.246067 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.311998 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.311932 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.644921 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.644848 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.745279 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.745214 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.832134 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.832099 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.896706 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.896626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.993719 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.993689 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j"] Sep 10 17:50:42.998639 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:42.998615 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.001290 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.001268 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 17:50:43.001385 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.001313 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kmxc8\"" Sep 10 17:50:43.008917 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.008891 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j"] Sep 10 17:50:43.110275 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.110244 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhrz6\" (UniqueName: \"kubernetes.io/projected/c347e8ae-f065-4b20-a978-129098798e99-kube-api-access-vhrz6\") pod \"payload-pre-processing-59fbc47bf8-ncr7j\" (UID: \"c347e8ae-f065-4b20-a978-129098798e99\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.110432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.110343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c347e8ae-f065-4b20-a978-129098798e99-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ncr7j\" (UID: \"c347e8ae-f065-4b20-a978-129098798e99\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.211284 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.211259 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhrz6\" (UniqueName: \"kubernetes.io/projected/c347e8ae-f065-4b20-a978-129098798e99-kube-api-access-vhrz6\") pod \"payload-pre-processing-59fbc47bf8-ncr7j\" (UID: \"c347e8ae-f065-4b20-a978-129098798e99\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.211406 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.211339 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c347e8ae-f065-4b20-a978-129098798e99-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ncr7j\" (UID: \"c347e8ae-f065-4b20-a978-129098798e99\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.211970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.211951 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c347e8ae-f065-4b20-a978-129098798e99-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ncr7j\" (UID: \"c347e8ae-f065-4b20-a978-129098798e99\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.219580 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.219552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhrz6\" (UniqueName: \"kubernetes.io/projected/c347e8ae-f065-4b20-a978-129098798e99-kube-api-access-vhrz6\") pod \"payload-pre-processing-59fbc47bf8-ncr7j\" (UID: \"c347e8ae-f065-4b20-a978-129098798e99\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.310583 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.310551 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:43.587545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:43.587514 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j"] Sep 10 17:50:44.282251 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:44.282195 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" event={"ID":"c347e8ae-f065-4b20-a978-129098798e99","Type":"ContainerStarted","Data":"3a96bb566b99c72ac8e3fdb61cc7816b641db213acf3233404df84f03d793c2b"} Sep 10 17:50:44.577009 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:44.576898 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:46.293284 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:46.293246 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" event={"ID":"c347e8ae-f065-4b20-a978-129098798e99","Type":"ContainerStarted","Data":"49617f3fd4808a2ed5d3a57cff1d34ae65902905ded207a3393b72c6dd4c2666"} Sep 10 17:50:46.293692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:46.293319 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:46.316617 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:46.312331 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" podStartSLOduration=2.199964604 podStartE2EDuration="4.312309755s" totalImagesPullingTime="2.112345151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:46.311098074 +0000 UTC m=+575.192823216" watchObservedRunningTime="2026-09-10 17:50:46.312309755 +0000 UTC m=+575.194034898" Sep 10 17:50:52.322533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:52.322493 2565 generic.go:356] "Generic (PLEG): container finished" podID="ac2ec7ce-2b19-4a7e-ba20-f88f490d804e" containerID="d7cf5494231a071fd9593d5fe238877d0383cde6889beaf18d5dfd40ff367e1c" exitCode=0 Sep 10 17:50:52.322938 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:52.322562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" event={"ID":"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e","Type":"ContainerDied","Data":"d7cf5494231a071fd9593d5fe238877d0383cde6889beaf18d5dfd40ff367e1c"} Sep 10 17:50:52.322938 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:52.322928 2565 scope.go:117] "RemoveContainer" containerID="d7cf5494231a071fd9593d5fe238877d0383cde6889beaf18d5dfd40ff367e1c" Sep 10 17:50:52.323912 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:52.323885 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:50:52.323994 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:52.323980 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:50:52.324037 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:52.324018 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:50:53.329114 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:53.329071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" event={"ID":"ac2ec7ce-2b19-4a7e-ba20-f88f490d804e","Type":"ContainerStarted","Data":"747fb0b110f76c97c3cff89c29487cc3c70892b046da64395f92b1b7fff1f257"} Sep 10 17:50:53.611936 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:53.611858 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:53.703649 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:53.703603 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:54.054941 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:54.054905 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:50:54.055333 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:54.055249 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-n7pc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Sep 10 17:50:54.055508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:54.055485 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" podUID="ac2ec7ce-2b19-4a7e-ba20-f88f490d804e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Sep 10 17:50:55.055739 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:55.055283 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-n7pc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Sep 10 17:50:55.055739 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:55.055357 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" podUID="ac2ec7ce-2b19-4a7e-ba20-f88f490d804e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Sep 10 17:50:56.054780 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:56.054734 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-n7pc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Sep 10 17:50:56.054981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:56.054805 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" podUID="ac2ec7ce-2b19-4a7e-ba20-f88f490d804e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Sep 10 17:50:57.054782 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:57.054734 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-n7pc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Sep 10 17:50:57.055280 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:57.054819 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" podUID="ac2ec7ce-2b19-4a7e-ba20-f88f490d804e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Sep 10 17:50:57.300414 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:57.300382 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ncr7j" Sep 10 17:50:58.055324 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:58.055278 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-n7pc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Sep 10 17:50:58.055815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:58.055354 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" podUID="ac2ec7ce-2b19-4a7e-ba20-f88f490d804e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Sep 10 17:50:59.058960 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:59.058931 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:50:59.059355 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:59.059206 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:50:59.060318 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:50:59.060292 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-n7pc7" Sep 10 17:51:05.784022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:05.783986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:20.712099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.712063 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7"] Sep 10 17:51:20.715779 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.715756 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.718755 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.718736 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 17:51:20.719956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.719928 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6fj4v\"" Sep 10 17:51:20.720416 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.720395 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 17:51:20.721187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.721141 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 17:51:20.727097 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.727074 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7"] Sep 10 17:51:20.751287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.751256 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn2p7\" (UniqueName: \"kubernetes.io/projected/eb58e274-72a5-4529-973c-8404c8d731f1-kube-api-access-xn2p7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.751392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.751368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb58e274-72a5-4529-973c-8404c8d731f1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.751436 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.751409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.751482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.751448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.751521 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.751497 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.751521 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.751517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.813098 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.813073 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:20.852055 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb58e274-72a5-4529-973c-8404c8d731f1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852068 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852161 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn2p7\" (UniqueName: \"kubernetes.io/projected/eb58e274-72a5-4529-973c-8404c8d731f1-kube-api-access-xn2p7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852631 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852608 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852631 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852623 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.852749 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.852648 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.854485 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.854463 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb58e274-72a5-4529-973c-8404c8d731f1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.854617 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.854601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb58e274-72a5-4529-973c-8404c8d731f1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:20.860968 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:20.860946 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn2p7\" (UniqueName: \"kubernetes.io/projected/eb58e274-72a5-4529-973c-8404c8d731f1-kube-api-access-xn2p7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7\" (UID: \"eb58e274-72a5-4529-973c-8404c8d731f1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:21.027062 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:21.026986 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:21.298716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:21.298690 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7"] Sep 10 17:51:21.300592 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:51:21.300564 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb58e274_72a5_4529_973c_8404c8d731f1.slice/crio-7f1154823b19c289e4ae6bb64dbe426b9ce750d878f06a40de1db1e955fa9df7 WatchSource:0}: Error finding container 7f1154823b19c289e4ae6bb64dbe426b9ce750d878f06a40de1db1e955fa9df7: Status 404 returned error can't find the container with id 7f1154823b19c289e4ae6bb64dbe426b9ce750d878f06a40de1db1e955fa9df7 Sep 10 17:51:21.463146 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:21.463117 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" event={"ID":"eb58e274-72a5-4529-973c-8404c8d731f1","Type":"ContainerStarted","Data":"7f1154823b19c289e4ae6bb64dbe426b9ce750d878f06a40de1db1e955fa9df7"} Sep 10 17:51:22.469186 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:22.469154 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" event={"ID":"eb58e274-72a5-4529-973c-8404c8d731f1","Type":"ContainerStarted","Data":"0a8248663c42e24d1f04469fb4c3132faebb13c23bacea49ac0261162f735ad6"} Sep 10 17:51:22.490305 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:22.490260 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" podStartSLOduration=1.5094335779999999 podStartE2EDuration="2.490244365s" totalImagesPullingTime="980.810787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:22.489700021 +0000 UTC m=+611.371425161" watchObservedRunningTime="2026-09-10 17:51:22.490244365 +0000 UTC m=+611.371969507" Sep 10 17:51:25.853113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:25.853082 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:31.027779 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:31.027742 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:31.028307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:31.027813 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:31.040066 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:31.040038 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:31.526311 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:31.526284 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7" Sep 10 17:51:32.697975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.697927 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2"] Sep 10 17:51:32.701743 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.701722 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.704684 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.704664 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 10 17:51:32.712253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.712211 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2"] Sep 10 17:51:32.726552 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.726531 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:32.846878 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.846853 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.846989 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.846886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m295j\" (UniqueName: \"kubernetes.io/projected/3407dc1e-fc6a-48be-a623-d0baa0902cec-kube-api-access-m295j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.846989 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.846910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.847095 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.847017 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.847095 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.847054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.847160 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.847095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3407dc1e-fc6a-48be-a623-d0baa0902cec-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948405 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948405 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948374 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m295j\" (UniqueName: \"kubernetes.io/projected/3407dc1e-fc6a-48be-a623-d0baa0902cec-kube-api-access-m295j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948405 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948393 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948627 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948627 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948627 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948488 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3407dc1e-fc6a-48be-a623-d0baa0902cec-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948785 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948810 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.948943 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.948843 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.950766 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.950739 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3407dc1e-fc6a-48be-a623-d0baa0902cec-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.951045 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.951026 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3407dc1e-fc6a-48be-a623-d0baa0902cec-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:32.956625 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:32.956601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m295j\" (UniqueName: \"kubernetes.io/projected/3407dc1e-fc6a-48be-a623-d0baa0902cec-kube-api-access-m295j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wslb2\" (UID: \"3407dc1e-fc6a-48be-a623-d0baa0902cec\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:33.012934 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:33.012912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:33.288209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:33.288128 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2"] Sep 10 17:51:33.292944 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:51:33.292918 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3407dc1e_fc6a_48be_a623_d0baa0902cec.slice/crio-527fde72d6b089b154767e106f5354c81f49563e36dd34cd1ac243b108669acf WatchSource:0}: Error finding container 527fde72d6b089b154767e106f5354c81f49563e36dd34cd1ac243b108669acf: Status 404 returned error can't find the container with id 527fde72d6b089b154767e106f5354c81f49563e36dd34cd1ac243b108669acf Sep 10 17:51:33.294258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:33.294218 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:51:33.526440 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:33.526414 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" event={"ID":"3407dc1e-fc6a-48be-a623-d0baa0902cec","Type":"ContainerStarted","Data":"527fde72d6b089b154767e106f5354c81f49563e36dd34cd1ac243b108669acf"} Sep 10 17:51:34.533686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:34.533611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" event={"ID":"3407dc1e-fc6a-48be-a623-d0baa0902cec","Type":"ContainerStarted","Data":"9522e7febabdd17f332bc28a53ebbdeb82fb72e064d39bed39f6f2f8c89a6d7e"} Sep 10 17:51:34.553441 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:34.553391 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" podStartSLOduration=2.375934457 podStartE2EDuration="2.553374978s" totalImagesPullingTime="177.440521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:34.551793474 +0000 UTC m=+623.433518616" watchObservedRunningTime="2026-09-10 17:51:34.553374978 +0000 UTC m=+623.435100119" Sep 10 17:51:39.223675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:39.223644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:43.013527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:43.013494 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:43.013929 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:43.013554 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:43.026062 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:43.026033 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:43.587903 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:43.587874 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wslb2" Sep 10 17:51:51.235271 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:51:51.235194 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:05.735429 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:05.735388 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:14.463725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:14.463652 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:14.564903 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:14.564850 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:20.350382 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:20.350340 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:28.907166 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:28.907131 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59"] Sep 10 17:52:28.910801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:28.910785 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:28.914701 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:28.914662 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 17:52:28.922903 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:28.922879 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59"] Sep 10 17:52:28.925898 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:28.925875 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:29.023560 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.023518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.023725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.023568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.023725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.023603 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.023725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.023674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcbhm\" (UniqueName: \"kubernetes.io/projected/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-kube-api-access-dcbhm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.023725 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.023712 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.023923 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.023738 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.124627 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.124585 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.124801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.124632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.124801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.124672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.124801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.124720 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcbhm\" (UniqueName: \"kubernetes.io/projected/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-kube-api-access-dcbhm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.124972 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.124874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.124972 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.124934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.125235 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.125196 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.125334 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.125272 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.125334 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.125304 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.127041 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.127008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.127169 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.127149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.132809 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.132792 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcbhm\" (UniqueName: \"kubernetes.io/projected/70e42ecd-0b24-4128-9e88-b546c7ce0ad0-kube-api-access-dcbhm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rkr59\" (UID: \"70e42ecd-0b24-4128-9e88-b546c7ce0ad0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.224831 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.224796 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:29.503533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.503502 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59"] Sep 10 17:52:29.505407 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:52:29.505382 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e42ecd_0b24_4128_9e88_b546c7ce0ad0.slice/crio-66eb859403671329524cd49120bd8bafad3e6ec800b36b71661c170b2a9f02eb WatchSource:0}: Error finding container 66eb859403671329524cd49120bd8bafad3e6ec800b36b71661c170b2a9f02eb: Status 404 returned error can't find the container with id 66eb859403671329524cd49120bd8bafad3e6ec800b36b71661c170b2a9f02eb Sep 10 17:52:29.801849 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.801756 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" event={"ID":"70e42ecd-0b24-4128-9e88-b546c7ce0ad0","Type":"ContainerStarted","Data":"0dd97fb1e27d06e3b57205cbca84ca3c2630048cc61afd5f7b58fc9c25ea8a9e"} Sep 10 17:52:29.801849 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.801820 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" event={"ID":"70e42ecd-0b24-4128-9e88-b546c7ce0ad0","Type":"ContainerStarted","Data":"66eb859403671329524cd49120bd8bafad3e6ec800b36b71661c170b2a9f02eb"} Sep 10 17:52:29.821066 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:29.821012 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" podStartSLOduration=1.642440776 podStartE2EDuration="1.82099688s" totalImagesPullingTime="178.556104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:29.818859646 +0000 UTC m=+678.700584793" watchObservedRunningTime="2026-09-10 17:52:29.82099688 +0000 UTC m=+678.702722021" Sep 10 17:52:32.194467 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:32.194429 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:39.145270 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:39.145210 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:39.225880 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:39.225841 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:39.226037 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:39.225911 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:39.238536 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:39.238513 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:52:39.865008 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:52:39.864971 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rkr59" Sep 10 17:53:25.990285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:53:25.990181 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:07.961471 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:07.961436 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:09.065718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:09.065653 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:09.183112 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:09.183073 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:10.203401 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:10.203371 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:10.302219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:10.302187 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:19.438180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:19.438140 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:25.599463 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.599431 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:25.728441 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.728409 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:25.920746 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.920714 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl"] Sep 10 17:54:25.924325 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.924300 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:25.926832 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.926810 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c34c13-openshift-default-dockercfg-9m9zs\"" Sep 10 17:54:25.935678 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.935653 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl"] Sep 10 17:54:25.938376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:25.938350 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:26.018649 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018607 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krr9j\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-kube-api-access-krr9j\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.018834 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018661 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-socket\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.018834 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018761 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-envoy\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.018957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istiod-ca-cert\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.018957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-token\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.019065 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018955 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-certs\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.019065 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.018978 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-data\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.019065 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.019021 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-credential-socket\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.019065 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.019055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-podinfo\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120520 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-podinfo\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120531 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krr9j\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-kube-api-access-krr9j\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120554 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-socket\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120584 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-envoy\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istiod-ca-cert\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-token\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-certs\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-data\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.120904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.120798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-credential-socket\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.121142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.121115 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-socket\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.121376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.121341 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-credential-socket\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.121376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.121354 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-data\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.121475 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.121457 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-certs\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.121566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.121542 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istiod-ca-cert\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.123322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.123292 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-envoy\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.123472 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.123458 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-podinfo\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.129423 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.129401 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krr9j\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-kube-api-access-krr9j\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.129423 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.129416 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-token\") pod \"e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.236365 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.236311 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:26.363124 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.363079 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:26.512322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.512292 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl"] Sep 10 17:54:26.514333 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:54:26.514304 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc9f3d1_90a1_4a6a_9350_40f3611b6326.slice/crio-894b8212b73a8effb8c5dc3986a46cca8e0ba082770f01bb3c313cf8aba30893 WatchSource:0}: Error finding container 894b8212b73a8effb8c5dc3986a46cca8e0ba082770f01bb3c313cf8aba30893: Status 404 returned error can't find the container with id 894b8212b73a8effb8c5dc3986a46cca8e0ba082770f01bb3c313cf8aba30893 Sep 10 17:54:26.516485 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.516452 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:26.516592 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.516529 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:26.516592 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:26.516574 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:27.042085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:27.042052 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:27.137904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:27.137874 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:27.387761 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:27.387680 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" event={"ID":"4cc9f3d1-90a1-4a6a-9350-40f3611b6326","Type":"ContainerStarted","Data":"a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509"} Sep 10 17:54:27.387904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:27.387766 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" event={"ID":"4cc9f3d1-90a1-4a6a-9350-40f3611b6326","Type":"ContainerStarted","Data":"894b8212b73a8effb8c5dc3986a46cca8e0ba082770f01bb3c313cf8aba30893"} Sep 10 17:54:27.408859 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:27.408808 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podStartSLOduration=2.408792226 podStartE2EDuration="2.408792226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:27.408471827 +0000 UTC m=+796.290196972" watchObservedRunningTime="2026-09-10 17:54:27.408792226 +0000 UTC m=+796.290517368" Sep 10 17:54:27.512719 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:27.512690 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:28.237286 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:28.237250 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:28.242132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:28.242108 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:28.392799 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:28.392770 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:28.393625 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:28.393602 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:29.404744 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:29.404713 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:29.505168 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:29.505131 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:30.723642 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:30.723607 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:30.820980 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:30.820947 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:31.181031 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:31.180991 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:31.280138 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:31.280107 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:31.350813 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:31.350778 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:33.386788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.386756 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:33.489032 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.488998 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:33.626269 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.626234 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:33.885924 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.885846 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl"] Sep 10 17:54:33.891111 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.891091 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.893828 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.893800 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-007ef522-openshift-default-dockercfg-prwzr\"" Sep 10 17:54:33.902049 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.902027 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl"] Sep 10 17:54:33.906635 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.906613 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:33.994698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.994646 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-certs\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995045 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-credential-socket\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995078 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-envoy\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995292 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995123 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96dbae1-3e84-4158-94f5-fb388432669f-istio-podinfo\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995292 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995165 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-data\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995292 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995193 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-socket\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995292 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96dbae1-3e84-4158-94f5-fb388432669f-istiod-ca-cert\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995516 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-istio-token\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:33.995516 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:33.995389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77c5n\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-kube-api-access-77c5n\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.034808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.034782 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:34.096165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096139 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96dbae1-3e84-4158-94f5-fb388432669f-istio-podinfo\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-data\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096194 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-socket\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96dbae1-3e84-4158-94f5-fb388432669f-istiod-ca-cert\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-istio-token\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77c5n\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-kube-api-access-77c5n\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096360 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-certs\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096384 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-credential-socket\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096408 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-envoy\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096750 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096600 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-data\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096750 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096656 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-socket\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096883 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-certs\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.096883 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096876 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-credential-socket\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.097001 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.096951 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96dbae1-3e84-4158-94f5-fb388432669f-istiod-ca-cert\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.098708 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.098687 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-envoy\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.098825 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.098806 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96dbae1-3e84-4158-94f5-fb388432669f-istio-podinfo\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.111257 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.111213 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77c5n\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-kube-api-access-77c5n\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.111338 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.111312 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-istio-token\") pod \"e2e-ait-007ef522-openshift-default-749747b95d-pkzbl\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.203685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.203663 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:34.321713 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.321679 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk"] Sep 10 17:54:34.325960 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.325930 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.329060 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.329038 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-bcdd7c-openshift-default-dockercfg-khvmv\"" Sep 10 17:54:34.336665 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.336644 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk"] Sep 10 17:54:34.337944 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.337915 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:34.443129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.443095 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:34.487275 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.487253 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl"] Sep 10 17:54:34.489520 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:54:34.489497 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96dbae1_3e84_4158_94f5_fb388432669f.slice/crio-bfbf5b7e6e76d6524c99cea5f07ee5ff83543d3312f8a902cdbf31aa06642993 WatchSource:0}: Error finding container bfbf5b7e6e76d6524c99cea5f07ee5ff83543d3312f8a902cdbf31aa06642993: Status 404 returned error can't find the container with id bfbf5b7e6e76d6524c99cea5f07ee5ff83543d3312f8a902cdbf31aa06642993 Sep 10 17:54:34.491605 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.491576 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:34.491681 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.491636 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:34.491681 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.491664 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:34.500545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500523 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-socket\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-credential-socket\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500691 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cd9904-941c-4737-baa8-f9d71103d27a-istiod-ca-cert\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500702 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-data\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500750 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-certs\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500820 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500789 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-envoy\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500867 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500832 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cd9904-941c-4737-baa8-f9d71103d27a-istio-podinfo\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500914 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff6zp\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-kube-api-access-ff6zp\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.500977 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.500949 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-istio-token\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.501163 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.501145 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:34.601708 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.601670 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-data\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.601708 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.601711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-certs\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.601944 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.601829 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-envoy\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.601944 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.601890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cd9904-941c-4737-baa8-f9d71103d27a-istio-podinfo\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.601944 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.601934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff6zp\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-kube-api-access-ff6zp\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.601981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-istio-token\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602035 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-certs\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602074 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-data\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602115 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-socket\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602383 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602147 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-credential-socket\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602383 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602211 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cd9904-941c-4737-baa8-f9d71103d27a-istiod-ca-cert\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602461 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-credential-socket\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602538 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-socket\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.602815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.602797 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cd9904-941c-4737-baa8-f9d71103d27a-istiod-ca-cert\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.604395 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.604376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cd9904-941c-4737-baa8-f9d71103d27a-istio-podinfo\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.604515 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.604497 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-envoy\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.610535 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.610515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-istio-token\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.610738 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.610717 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff6zp\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-kube-api-access-ff6zp\") pod \"e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.638957 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.638927 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:34.918830 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.918805 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk"] Sep 10 17:54:34.920293 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:54:34.920267 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cd9904_941c_4737_baa8_f9d71103d27a.slice/crio-5320135b2da715d75d0542b0479db0f5b1d81669efeccef2d86364d54eb331c3 WatchSource:0}: Error finding container 5320135b2da715d75d0542b0479db0f5b1d81669efeccef2d86364d54eb331c3: Status 404 returned error can't find the container with id 5320135b2da715d75d0542b0479db0f5b1d81669efeccef2d86364d54eb331c3 Sep 10 17:54:34.922535 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.922500 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:34.922598 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.922571 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:34.922644 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:34.922599 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:35.436022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.435972 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" event={"ID":"d96dbae1-3e84-4158-94f5-fb388432669f","Type":"ContainerStarted","Data":"5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f"} Sep 10 17:54:35.436217 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.436043 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" event={"ID":"d96dbae1-3e84-4158-94f5-fb388432669f","Type":"ContainerStarted","Data":"bfbf5b7e6e76d6524c99cea5f07ee5ff83543d3312f8a902cdbf31aa06642993"} Sep 10 17:54:35.437565 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.437540 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" event={"ID":"78cd9904-941c-4737-baa8-f9d71103d27a","Type":"ContainerStarted","Data":"0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49"} Sep 10 17:54:35.437666 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.437580 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" event={"ID":"78cd9904-941c-4737-baa8-f9d71103d27a","Type":"ContainerStarted","Data":"5320135b2da715d75d0542b0479db0f5b1d81669efeccef2d86364d54eb331c3"} Sep 10 17:54:35.460709 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.460660 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podStartSLOduration=2.460644648 podStartE2EDuration="2.460644648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:35.458896543 +0000 UTC m=+804.340621708" watchObservedRunningTime="2026-09-10 17:54:35.460644648 +0000 UTC m=+804.342369789" Sep 10 17:54:35.482026 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.481980 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podStartSLOduration=1.481967086 podStartE2EDuration="1.481967086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:35.480588184 +0000 UTC m=+804.362313529" watchObservedRunningTime="2026-09-10 17:54:35.481967086 +0000 UTC m=+804.363692221" Sep 10 17:54:35.639896 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.639863 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:35.645088 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:35.645065 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:36.204270 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.204219 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:36.209057 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.209030 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:36.442647 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.442609 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:36.442828 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.442659 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:36.443338 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.443313 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:36.443652 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.443638 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:36.847462 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.847418 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:36.919352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:36.919320 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:37.350416 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:37.350387 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:37.446950 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:37.446917 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:38.098730 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:38.098697 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:38.202310 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:38.202270 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:38.281181 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:38.281148 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:38.345777 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:38.345739 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:38.924112 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:38.924079 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:39.024937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:39.024891 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:39.344986 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:39.344905 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:39.441692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:39.441660 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:39.676861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:39.676826 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:39.913170 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:39.913139 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:40.015914 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.015828 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:40.049536 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.049507 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2"] Sep 10 17:54:40.068105 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.068067 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.071252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.070199 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2"] Sep 10 17:54:40.072117 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.071845 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-bcdd7c-dockercfg-qjq99\"" Sep 10 17:54:40.072461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.072037 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-bcdd7c\"" Sep 10 17:54:40.169267 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.169206 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-plugins-config-volume\") pod \"payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.169466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.169360 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vclnv\" (UniqueName: \"kubernetes.io/projected/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-kube-api-access-vclnv\") pod \"payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.270036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.269951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-plugins-config-volume\") pod \"payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.270347 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.270322 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vclnv\" (UniqueName: \"kubernetes.io/projected/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-kube-api-access-vclnv\") pod \"payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.271715 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.271293 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-plugins-config-volume\") pod \"payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.279857 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.279832 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vclnv\" (UniqueName: \"kubernetes.io/projected/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-kube-api-access-vclnv\") pod \"payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.387402 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.387375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:40.448983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.448884 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:40.660219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:40.660132 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2"] Sep 10 17:54:41.130426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:41.130388 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:41.485681 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:41.485641 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" event={"ID":"d9dd25c5-c6c7-4275-b41e-c58399bf20e3","Type":"ContainerStarted","Data":"8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024"} Sep 10 17:54:41.486163 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:41.485698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" event={"ID":"d9dd25c5-c6c7-4275-b41e-c58399bf20e3","Type":"ContainerStarted","Data":"6887edc9c4f693b19fd2cd1c0841a596fd535f31bdfa2d7ada0acab124b045ce"} Sep 10 17:54:41.486163 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:41.485757 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:41.503367 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:41.503317 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" podStartSLOduration=1.503303033 podStartE2EDuration="1.503303033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:41.502185807 +0000 UTC m=+810.383910952" watchObservedRunningTime="2026-09-10 17:54:41.503303033 +0000 UTC m=+810.385028174" Sep 10 17:54:46.209116 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:46.209061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:46.345371 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:46.345327 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:46.438140 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:46.438107 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:46.619481 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:46.619363 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:48.528861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:48.528830 2565 generic.go:356] "Generic (PLEG): container finished" podID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerID="a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509" exitCode=0 Sep 10 17:54:48.529322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:48.528871 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" event={"ID":"4cc9f3d1-90a1-4a6a-9350-40f3611b6326","Type":"ContainerDied","Data":"a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509"} Sep 10 17:54:48.529322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:48.529303 2565 scope.go:117] "RemoveContainer" containerID="a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509" Sep 10 17:54:48.530402 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:48.530369 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:48.530522 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:48.530475 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:48.530587 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:48.530525 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:49.226543 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:49.226495 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:49.537072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:49.537036 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" event={"ID":"4cc9f3d1-90a1-4a6a-9350-40f3611b6326","Type":"ContainerStarted","Data":"2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449"} Sep 10 17:54:50.237493 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:50.237456 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:50.237674 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:50.237552 2565 patch_prober.go:28] interesting pod/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Sep 10 17:54:50.237674 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:50.237589 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Sep 10 17:54:51.236994 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:51.236945 2565 patch_prober.go:28] interesting pod/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Sep 10 17:54:51.237393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:51.237010 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Sep 10 17:54:51.642600 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:51.642518 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:51.743350 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:51.743320 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:52.237634 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:52.237594 2565 patch_prober.go:28] interesting pod/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Sep 10 17:54:52.237969 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:52.237648 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Sep 10 17:54:52.494064 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:52.493756 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:54:53.236853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:53.236810 2565 patch_prober.go:28] interesting pod/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Sep 10 17:54:53.237045 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:53.236887 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Sep 10 17:54:54.237705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.237666 2565 patch_prober.go:28] interesting pod/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Sep 10 17:54:54.238095 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.237729 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Sep 10 17:54:54.260090 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.260048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:54.584865 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.584778 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:54.880312 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.877480 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp"] Sep 10 17:54:54.887531 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.887505 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:54.890531 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.890501 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-18490e-openshift-default-dockercfg-chh62\"" Sep 10 17:54:54.901098 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.901069 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp"] Sep 10 17:54:54.914463 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:54.914430 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:55.046328 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8d7z\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-kube-api-access-f8d7z\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046520 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046404 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-socket\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046520 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5573d4b-300d-4179-81f5-108f34307c9d-istiod-ca-cert\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046652 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046537 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-certs\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046652 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046585 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-envoy\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046652 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-credential-socket\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046796 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5573d4b-300d-4179-81f5-108f34307c9d-istio-podinfo\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046796 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046769 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-data\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.046901 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.046811 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-istio-token\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.101368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.101330 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:55.148599 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-certs\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.148599 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-envoy\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.148842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148606 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-credential-socket\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.148842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148647 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5573d4b-300d-4179-81f5-108f34307c9d-istio-podinfo\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.148842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-data\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.148842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-istio-token\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.148842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8d7z\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-kube-api-access-f8d7z\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.149112 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-socket\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.149112 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.148906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5573d4b-300d-4179-81f5-108f34307c9d-istiod-ca-cert\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.149217 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.149121 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-credential-socket\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.149395 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.149340 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-certs\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.149528 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.149491 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-socket\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.149528 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.149503 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-data\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.150351 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.150327 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5573d4b-300d-4179-81f5-108f34307c9d-istiod-ca-cert\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.152175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.152151 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5573d4b-300d-4179-81f5-108f34307c9d-istio-podinfo\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.152819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.152797 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-envoy\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.162039 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.161993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8d7z\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-kube-api-access-f8d7z\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.162738 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.162709 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-istio-token\") pod \"e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.205904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.205865 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:55.442353 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.442320 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:55.443142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.443052 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:55.444580 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.444550 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:54:55.499684 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:54:55.498796 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5573d4b_300d_4179_81f5_108f34307c9d.slice/crio-2894eee1a670f748817d4542f16a022b1fee7c0850844efb0a88f192c3a8a66c WatchSource:0}: Error finding container 2894eee1a670f748817d4542f16a022b1fee7c0850844efb0a88f192c3a8a66c: Status 404 returned error can't find the container with id 2894eee1a670f748817d4542f16a022b1fee7c0850844efb0a88f192c3a8a66c Sep 10 17:54:55.501898 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.501840 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:55.502021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.501936 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:55.502021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.501980 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:55.505336 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.505308 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp"] Sep 10 17:54:55.582583 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:55.582557 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" event={"ID":"d5573d4b-300d-4179-81f5-108f34307c9d","Type":"ContainerStarted","Data":"2894eee1a670f748817d4542f16a022b1fee7c0850844efb0a88f192c3a8a66c"} Sep 10 17:54:56.590869 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.590836 2565 generic.go:356] "Generic (PLEG): container finished" podID="78cd9904-941c-4737-baa8-f9d71103d27a" containerID="0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49" exitCode=0 Sep 10 17:54:56.591316 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.590905 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" event={"ID":"78cd9904-941c-4737-baa8-f9d71103d27a","Type":"ContainerDied","Data":"0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49"} Sep 10 17:54:56.591411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.591393 2565 scope.go:117] "RemoveContainer" containerID="0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49" Sep 10 17:54:56.592672 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.592638 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:56.592781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.592682 2565 generic.go:356] "Generic (PLEG): container finished" podID="d96dbae1-3e84-4158-94f5-fb388432669f" containerID="5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f" exitCode=0 Sep 10 17:54:56.592781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.592722 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:56.592781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.592734 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" event={"ID":"d96dbae1-3e84-4158-94f5-fb388432669f","Type":"ContainerDied","Data":"5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f"} Sep 10 17:54:56.592781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.592764 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:56.593192 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.593167 2565 scope.go:117] "RemoveContainer" containerID="5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f" Sep 10 17:54:56.594442 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.594410 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:56.594675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.594491 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:56.594675 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.594538 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:54:56.594773 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.594732 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" event={"ID":"d5573d4b-300d-4179-81f5-108f34307c9d","Type":"ContainerStarted","Data":"2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a"} Sep 10 17:54:56.650547 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:56.650489 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podStartSLOduration=2.6504672129999998 podStartE2EDuration="2.650467213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:56.649708004 +0000 UTC m=+825.531433145" watchObservedRunningTime="2026-09-10 17:54:56.650467213 +0000 UTC m=+825.532192354" Sep 10 17:54:57.206336 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.206294 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:57.213119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.213091 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:57.602595 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.602498 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" event={"ID":"d96dbae1-3e84-4158-94f5-fb388432669f","Type":"ContainerStarted","Data":"3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80"} Sep 10 17:54:57.604728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.604704 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" event={"ID":"78cd9904-941c-4737-baa8-f9d71103d27a","Type":"ContainerStarted","Data":"c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564"} Sep 10 17:54:57.604992 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.604970 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:57.605823 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.605804 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:54:57.639838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.639804 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:54:57.640076 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.640043 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:54:57.640174 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:57.640103 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:54:58.204442 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.204405 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:54:58.204637 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.204616 2565 patch_prober.go:28] interesting pod/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 17:54:58.204721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.204674 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 17:54:58.512677 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.512564 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:58.608485 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.608457 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:58.640031 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.639996 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:54:58.640190 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:58.640049 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:54:59.204561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.204510 2565 patch_prober.go:28] interesting pod/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 17:54:59.204779 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.204571 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 17:54:59.329016 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.328977 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:59.640369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.640280 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:54:59.640369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.640337 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:54:59.754620 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.754580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:59.853619 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.853579 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:54:59.973893 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:54:59.973788 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:00.204945 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.204910 2565 patch_prober.go:28] interesting pod/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 17:55:00.205129 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.204969 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 17:55:00.321397 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.321308 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:00.420013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.419977 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:00.519425 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.519382 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:00.640338 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.640260 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:55:00.640338 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.640325 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:55:00.793250 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.791257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:00.892272 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:00.892154 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:01.204848 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:01.204796 2565 patch_prober.go:28] interesting pod/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 17:55:01.205024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:01.204860 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 17:55:01.222935 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:01.222913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:01.639417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:01.639318 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:55:01.639417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:01.639379 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:55:01.999845 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:01.999805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:02.204801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:02.204754 2565 patch_prober.go:28] interesting pod/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 17:55:02.204980 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:02.204834 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 17:55:02.639468 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:02.639435 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:55:02.639635 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:02.639489 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:55:02.938078 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:02.938050 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:03.204324 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:03.204196 2565 patch_prober.go:28] interesting pod/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 17:55:03.204847 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:03.204816 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 17:55:03.611252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:03.611145 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:03.639390 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:03.639362 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Sep 10 17:55:03.639605 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:03.639582 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Sep 10 17:55:04.336700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.336664 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:55:04.337290 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.337267 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:55:04.338570 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.338546 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:55:04.426700 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.426659 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:04.643904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.643830 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:55:04.644251 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.644199 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:55:04.644857 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:04.644839 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:55:06.373020 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:06.372983 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:06.482413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:06.482380 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:06.572438 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:06.572404 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:06.693677 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:06.693647 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:07.422213 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:07.422165 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:12.698474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:12.698436 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:12.798190 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:12.798142 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:18.458048 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.458000 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:18.723935 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.723852 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5573d4b-300d-4179-81f5-108f34307c9d" containerID="2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a" exitCode=0 Sep 10 17:55:18.724089 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.723923 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" event={"ID":"d5573d4b-300d-4179-81f5-108f34307c9d","Type":"ContainerDied","Data":"2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a"} Sep 10 17:55:18.724371 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.724357 2565 scope.go:117] "RemoveContainer" containerID="2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a" Sep 10 17:55:18.725427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.725395 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:18.725525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.725471 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:18.725525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:18.725502 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:19.027752 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.027658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.087683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.087645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.140593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.140562 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.219671 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.219639 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.278686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.278586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.341254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.341211 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl"] Sep 10 17:55:19.341511 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.341486 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" containerID="cri-o://3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80" gracePeriod=30 Sep 10 17:55:19.366093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.365988 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.730608 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.730572 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" event={"ID":"d5573d4b-300d-4179-81f5-108f34307c9d","Type":"ContainerStarted","Data":"c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a"} Sep 10 17:55:19.813255 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.813180 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:19.906791 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:19.906752 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:20.014609 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.014510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:20.206693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.206657 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:55:20.206951 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.206932 2565 patch_prober.go:28] interesting pod/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 10 17:55:20.207006 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.206988 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 10 17:55:20.300186 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.300095 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk"] Sep 10 17:55:20.303013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.302988 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.307955 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.307931 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-826cc2b1-gw-openshift-default-dockercfg-tcdww\"" Sep 10 17:55:20.320309 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.320283 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk"] Sep 10 17:55:20.337946 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.337922 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:20.440801 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.440761 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-certs\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.440970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.440816 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w749x\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-kube-api-access-w749x\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.440970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.440883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-data\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.440970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.440942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-socket\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.440970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.440959 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a704794-30a3-4193-9f93-b110caee61f0-istio-podinfo\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.441113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.440975 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a704794-30a3-4193-9f93-b110caee61f0-istiod-ca-cert\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.441113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.441063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-envoy\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.441113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.441086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-credential-socket\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.441113 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.441103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-istio-token\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.541705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541668 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-credential-socket\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.541705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-istio-token\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.541937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-certs\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.541937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541831 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w749x\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-kube-api-access-w749x\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.541937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541865 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-data\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.541937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-socket\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542160 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a704794-30a3-4193-9f93-b110caee61f0-istio-podinfo\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542160 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.541977 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a704794-30a3-4193-9f93-b110caee61f0-istiod-ca-cert\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542160 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.542076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-envoy\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542160 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.542131 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-credential-socket\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.542216 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-certs\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.542335 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-data\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.542380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-socket\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.542726 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.542702 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a704794-30a3-4193-9f93-b110caee61f0-istiod-ca-cert\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.544194 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.544168 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-envoy\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.544477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.544458 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a704794-30a3-4193-9f93-b110caee61f0-istio-podinfo\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.553209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.553146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w749x\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-kube-api-access-w749x\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.556669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.556643 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-istio-token\") pod \"e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.614539 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.614507 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:20.623365 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.623319 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:20.920600 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.920574 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk"] Sep 10 17:55:20.922831 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:55:20.922806 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a704794_30a3_4193_9f93_b110caee61f0.slice/crio-540ca90240a1177eab56472b30b4369e0f84541020b903bccaf34177458b8370 WatchSource:0}: Error finding container 540ca90240a1177eab56472b30b4369e0f84541020b903bccaf34177458b8370: Status 404 returned error can't find the container with id 540ca90240a1177eab56472b30b4369e0f84541020b903bccaf34177458b8370 Sep 10 17:55:20.925133 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.925101 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:20.925194 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.925165 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:20.925261 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:20.925195 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:21.165812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:21.165759 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:21.206939 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:21.206854 2565 patch_prober.go:28] interesting pod/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 10 17:55:21.206939 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:21.206910 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 10 17:55:21.746889 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:21.746851 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" event={"ID":"3a704794-30a3-4193-9f93-b110caee61f0","Type":"ContainerStarted","Data":"468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6"} Sep 10 17:55:21.747093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:21.746915 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" event={"ID":"3a704794-30a3-4193-9f93-b110caee61f0","Type":"ContainerStarted","Data":"540ca90240a1177eab56472b30b4369e0f84541020b903bccaf34177458b8370"} Sep 10 17:55:21.778314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:21.778254 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podStartSLOduration=1.7782179729999998 podStartE2EDuration="1.778217973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:55:21.775876555 +0000 UTC m=+850.657601711" watchObservedRunningTime="2026-09-10 17:55:21.778217973 +0000 UTC m=+850.659943115" Sep 10 17:55:22.207133 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:22.207098 2565 patch_prober.go:28] interesting pod/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 10 17:55:22.207540 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:22.207162 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 10 17:55:22.615063 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:22.614977 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:22.619937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:22.619911 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:22.753161 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:22.753129 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:22.758706 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:22.758673 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:23.206277 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:23.206211 2565 patch_prober.go:28] interesting pod/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 10 17:55:23.206466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:23.206305 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 10 17:55:23.421278 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:23.421246 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:23.523136 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:23.523042 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:23.750172 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:23.750143 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:23.852014 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:23.851909 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:24.070127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.070094 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:24.206551 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.206518 2565 patch_prober.go:28] interesting pod/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 10 17:55:24.206720 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.206585 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 10 17:55:24.207252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.207215 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:24.304260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.304201 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:24.642014 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.641991 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:55:24.785744 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.785639 2565 generic.go:356] "Generic (PLEG): container finished" podID="d96dbae1-3e84-4158-94f5-fb388432669f" containerID="3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80" exitCode=0 Sep 10 17:55:24.785744 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.785733 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" Sep 10 17:55:24.785940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.785731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" event={"ID":"d96dbae1-3e84-4158-94f5-fb388432669f","Type":"ContainerDied","Data":"3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80"} Sep 10 17:55:24.785940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.785860 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl" event={"ID":"d96dbae1-3e84-4158-94f5-fb388432669f","Type":"ContainerDied","Data":"bfbf5b7e6e76d6524c99cea5f07ee5ff83543d3312f8a902cdbf31aa06642993"} Sep 10 17:55:24.785940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.785884 2565 scope.go:117] "RemoveContainer" containerID="3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80" Sep 10 17:55:24.791994 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.791970 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-envoy\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792013 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96dbae1-3e84-4158-94f5-fb388432669f-istiod-ca-cert\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792042 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-certs\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792065 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-istio-token\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792122 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96dbae1-3e84-4158-94f5-fb388432669f-istio-podinfo\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792363 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792165 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77c5n\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-kube-api-access-77c5n\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792363 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792212 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-credential-socket\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792431 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:55:24.792676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792465 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d96dbae1-3e84-4158-94f5-fb388432669f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:55:24.792676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792588 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:55:24.792803 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-socket\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.792969 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.792819 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:55:24.793132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.793100 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-data\") pod \"d96dbae1-3e84-4158-94f5-fb388432669f\" (UID: \"d96dbae1-3e84-4158-94f5-fb388432669f\") " Sep 10 17:55:24.793455 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.793432 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:55:24.793887 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.793867 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.794007 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.793994 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96dbae1-3e84-4158-94f5-fb388432669f-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.794108 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.794094 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.794214 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.794186 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.794214 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.794211 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.794719 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.794685 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:55:24.794875 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.794855 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-kube-api-access-77c5n" (OuterVolumeSpecName: "kube-api-access-77c5n") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "kube-api-access-77c5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:55:24.795052 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.795024 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d96dbae1-3e84-4158-94f5-fb388432669f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:55:24.795157 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.795079 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d96dbae1-3e84-4158-94f5-fb388432669f" (UID: "d96dbae1-3e84-4158-94f5-fb388432669f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:55:24.796208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.796187 2565 scope.go:117] "RemoveContainer" containerID="5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f" Sep 10 17:55:24.815658 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.815636 2565 scope.go:117] "RemoveContainer" containerID="3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80" Sep 10 17:55:24.815999 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:55:24.815978 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80\": container with ID starting with 3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80 not found: ID does not exist" containerID="3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80" Sep 10 17:55:24.816060 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.816007 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80"} err="failed to get container status \"3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80\": rpc error: code = NotFound desc = could not find container \"3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80\": container with ID starting with 3dd3965558a9f4837d43faa04f310759892bf1cecefc11982ce8c197dff77f80 not found: ID does not exist" Sep 10 17:55:24.816060 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.816026 2565 scope.go:117] "RemoveContainer" containerID="5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f" Sep 10 17:55:24.816319 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:55:24.816301 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f\": container with ID starting with 5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f not found: ID does not exist" containerID="5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f" Sep 10 17:55:24.816418 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.816323 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f"} err="failed to get container status \"5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f\": rpc error: code = NotFound desc = could not find container \"5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f\": container with ID starting with 5b8dfab4edb5561a03a028585ef222b00cb4bc3eed3b1930e3d604809f31b85f not found: ID does not exist" Sep 10 17:55:24.895908 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.895867 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96dbae1-3e84-4158-94f5-fb388432669f-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.895908 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.895907 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.895908 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.895922 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96dbae1-3e84-4158-94f5-fb388432669f-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:24.896187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:24.895937 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77c5n\" (UniqueName: \"kubernetes.io/projected/d96dbae1-3e84-4158-94f5-fb388432669f-kube-api-access-77c5n\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:55:25.109932 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:25.109899 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl"] Sep 10 17:55:25.114837 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:25.114803 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-007ef522-openshift-default-749747b95d-pkzbl"] Sep 10 17:55:25.210800 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:25.210767 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:55:25.211078 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:25.211059 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:55:25.211707 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:25.211688 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:55:25.624287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:25.624249 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" path="/var/lib/kubelet/pods/d96dbae1-3e84-4158-94f5-fb388432669f/volumes" Sep 10 17:55:45.890360 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:45.890328 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:46.911819 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:46.911733 2565 generic.go:356] "Generic (PLEG): container finished" podID="3a704794-30a3-4193-9f93-b110caee61f0" containerID="468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6" exitCode=0 Sep 10 17:55:46.912212 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:46.911807 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" event={"ID":"3a704794-30a3-4193-9f93-b110caee61f0","Type":"ContainerDied","Data":"468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6"} Sep 10 17:55:46.912286 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:46.912250 2565 scope.go:117] "RemoveContainer" containerID="468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6" Sep 10 17:55:46.913419 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:46.913389 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:46.913490 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:46.913473 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:46.913530 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:46.913508 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:47.918982 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:47.918945 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" event={"ID":"3a704794-30a3-4193-9f93-b110caee61f0","Type":"ContainerStarted","Data":"10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb"} Sep 10 17:55:48.614880 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:48.614843 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:48.615683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:48.615660 2565 patch_prober.go:28] interesting pod/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 10 17:55:48.615747 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:48.615709 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 10 17:55:49.615772 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:49.615736 2565 patch_prober.go:28] interesting pod/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 10 17:55:49.616166 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:49.615802 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 10 17:55:50.615772 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:50.615741 2565 patch_prober.go:28] interesting pod/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 10 17:55:50.615940 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:50.615799 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 10 17:55:51.615649 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:51.615610 2565 patch_prober.go:28] interesting pod/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 10 17:55:51.615810 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:51.615669 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 10 17:55:52.615458 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:52.615420 2565 patch_prober.go:28] interesting pod/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 10 17:55:52.615838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:52.615482 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 10 17:55:52.753966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:52.753924 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:53.622977 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:53.622948 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:53.624061 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:53.624042 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:55:54.965578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:54.965542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:55.024694 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.024654 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:55.082891 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.082860 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:55.213959 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.213913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:55.269499 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.269422 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:55.368937 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.368906 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk"] Sep 10 17:55:55.369127 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.369102 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" containerID="cri-o://10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb" gracePeriod=30 Sep 10 17:55:55.392504 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.392476 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:55.813317 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:55.813283 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:56.097037 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.096954 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v"] Sep 10 17:55:56.098423 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.098394 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" Sep 10 17:55:56.098593 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.098579 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" Sep 10 17:55:56.098733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.098720 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" Sep 10 17:55:56.098822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.098810 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" Sep 10 17:55:56.099150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.099132 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" Sep 10 17:55:56.099302 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.099288 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96dbae1-3e84-4158-94f5-fb388432669f" containerName="istio-proxy" Sep 10 17:55:56.102596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.102572 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.105723 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.105696 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9584e914-openshift-default-dockercfg-cqgk5\"" Sep 10 17:55:56.108877 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.108856 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v"] Sep 10 17:55:56.115883 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.115856 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:56.216400 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216372 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-certs\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216532 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216408 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-token\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216532 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-podinfo\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216609 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216540 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-data\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216609 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-credential-socket\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216609 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216595 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c213af-bd2a-482f-8365-9838d3ce5389-istiod-ca-cert\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mh7w\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-kube-api-access-7mh7w\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-envoy\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.216833 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.216741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-socket\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.299254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.299193 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:56.317526 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-credential-socket\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317526 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c213af-bd2a-482f-8365-9838d3ce5389-istiod-ca-cert\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mh7w\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-kube-api-access-7mh7w\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-envoy\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317592 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-socket\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317631 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-certs\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-token\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.317721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-podinfo\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.318025 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-data\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.318025 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.317946 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-credential-socket\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.318145 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.318100 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-certs\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.318272 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.318252 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c213af-bd2a-482f-8365-9838d3ce5389-istiod-ca-cert\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.318343 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.318329 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-data\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.318382 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.318328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-socket\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.319933 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.319913 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-envoy\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.320150 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.320133 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-podinfo\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.325783 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.325754 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-token\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.326193 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.326172 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mh7w\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-kube-api-access-7mh7w\") pod \"e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.416096 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.416015 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:56.690771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.690742 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v"] Sep 10 17:55:56.692187 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:55:56.692164 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c213af_bd2a_482f_8365_9838d3ce5389.slice/crio-bff832dc818a9c871026d1b12996a98a6377ea72513ac132a2a797e303f11c0f WatchSource:0}: Error finding container bff832dc818a9c871026d1b12996a98a6377ea72513ac132a2a797e303f11c0f: Status 404 returned error can't find the container with id bff832dc818a9c871026d1b12996a98a6377ea72513ac132a2a797e303f11c0f Sep 10 17:55:56.694557 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.694506 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:56.694661 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.694600 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:56.694661 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.694644 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:55:56.976307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.976200 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" event={"ID":"a2c213af-bd2a-482f-8365-9838d3ce5389","Type":"ContainerStarted","Data":"9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03"} Sep 10 17:55:56.976307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.976278 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" event={"ID":"a2c213af-bd2a-482f-8365-9838d3ce5389","Type":"ContainerStarted","Data":"bff832dc818a9c871026d1b12996a98a6377ea72513ac132a2a797e303f11c0f"} Sep 10 17:55:56.997525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:56.997478 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podStartSLOduration=0.997460984 podStartE2EDuration="997.460984ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:55:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:55:56.995594123 +0000 UTC m=+885.877319277" watchObservedRunningTime="2026-09-10 17:55:56.997460984 +0000 UTC m=+885.879186125" Sep 10 17:55:57.417271 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:57.417169 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:57.422380 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:57.422357 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:57.981970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:57.981938 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:57.982815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:57.982795 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:55:58.322990 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:58.322915 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:58.416512 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:58.416476 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:59.471641 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:59.471611 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:59.572159 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:59.572126 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:55:59.836686 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:55:59.836605 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:00.067180 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.067151 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:00.167314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.167234 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:00.629092 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.629071 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:56:00.764741 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764664 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-data\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.764905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764750 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w749x\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-kube-api-access-w749x\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.764905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764785 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-certs\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.764905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764806 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a704794-30a3-4193-9f93-b110caee61f0-istiod-ca-cert\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.764905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764823 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-istio-token\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.764905 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764868 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a704794-30a3-4193-9f93-b110caee61f0-istio-podinfo\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.765170 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764921 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-socket\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.765170 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764956 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-credential-socket\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.765170 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764968 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:00.765170 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.764985 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-envoy\") pod \"3a704794-30a3-4193-9f93-b110caee61f0\" (UID: \"3a704794-30a3-4193-9f93-b110caee61f0\") " Sep 10 17:56:00.765170 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765055 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:00.765446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765303 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:00.765446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765332 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a704794-30a3-4193-9f93-b110caee61f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:56:00.765555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765498 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.765555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765520 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.765555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765535 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.765555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765534 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:00.765555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.765548 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a704794-30a3-4193-9f93-b110caee61f0-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.767017 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.766994 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:56:00.767175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.767160 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:00.767403 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.767382 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a704794-30a3-4193-9f93-b110caee61f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:56:00.767465 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.767431 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-kube-api-access-w749x" (OuterVolumeSpecName: "kube-api-access-w749x") pod "3a704794-30a3-4193-9f93-b110caee61f0" (UID: "3a704794-30a3-4193-9f93-b110caee61f0"). InnerVolumeSpecName "kube-api-access-w749x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:56:00.866031 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.866001 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w749x\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-kube-api-access-w749x\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.866031 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.866026 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a704794-30a3-4193-9f93-b110caee61f0-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.866181 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.866039 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a704794-30a3-4193-9f93-b110caee61f0-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.866181 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.866050 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:00.866181 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:00.866061 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a704794-30a3-4193-9f93-b110caee61f0-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:01.004783 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.004752 2565 generic.go:356] "Generic (PLEG): container finished" podID="3a704794-30a3-4193-9f93-b110caee61f0" containerID="10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb" exitCode=0 Sep 10 17:56:01.004906 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.004806 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" Sep 10 17:56:01.004906 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.004838 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" event={"ID":"3a704794-30a3-4193-9f93-b110caee61f0","Type":"ContainerDied","Data":"10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb"} Sep 10 17:56:01.005046 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.004934 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk" event={"ID":"3a704794-30a3-4193-9f93-b110caee61f0","Type":"ContainerDied","Data":"540ca90240a1177eab56472b30b4369e0f84541020b903bccaf34177458b8370"} Sep 10 17:56:01.005046 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.004969 2565 scope.go:117] "RemoveContainer" containerID="10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb" Sep 10 17:56:01.014524 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.014505 2565 scope.go:117] "RemoveContainer" containerID="468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6" Sep 10 17:56:01.022374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.022358 2565 scope.go:117] "RemoveContainer" containerID="10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb" Sep 10 17:56:01.022620 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:56:01.022600 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb\": container with ID starting with 10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb not found: ID does not exist" containerID="10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb" Sep 10 17:56:01.022678 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.022627 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb"} err="failed to get container status \"10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb\": rpc error: code = NotFound desc = could not find container \"10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb\": container with ID starting with 10a110a2807b51efb74866497a5a9431b178a9ac3c1bde8ef9a17d29979232cb not found: ID does not exist" Sep 10 17:56:01.022678 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.022643 2565 scope.go:117] "RemoveContainer" containerID="468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6" Sep 10 17:56:01.022887 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:56:01.022870 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6\": container with ID starting with 468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6 not found: ID does not exist" containerID="468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6" Sep 10 17:56:01.022927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.022892 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6"} err="failed to get container status \"468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6\": rpc error: code = NotFound desc = could not find container \"468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6\": container with ID starting with 468b3e14aa04e7d284129d73d1a798cd4f6731c35eedf3df3ec258f9820536a6 not found: ID does not exist" Sep 10 17:56:01.027876 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.027856 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk"] Sep 10 17:56:01.034039 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.034015 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-826cc2b1-gw-openshift-default-86df49587b-j28jk"] Sep 10 17:56:01.625543 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:01.625506 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a704794-30a3-4193-9f93-b110caee61f0" path="/var/lib/kubelet/pods/3a704794-30a3-4193-9f93-b110caee61f0/volumes" Sep 10 17:56:04.228208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:04.228120 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:10.804942 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:10.804901 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:20.529483 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:20.529442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:22.144107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:22.144069 2565 generic.go:356] "Generic (PLEG): container finished" podID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerID="9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03" exitCode=0 Sep 10 17:56:22.144582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:22.144145 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" event={"ID":"a2c213af-bd2a-482f-8365-9838d3ce5389","Type":"ContainerDied","Data":"9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03"} Sep 10 17:56:22.144582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:22.144565 2565 scope.go:117] "RemoveContainer" containerID="9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03" Sep 10 17:56:22.145608 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:22.145581 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:22.145661 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:22.145650 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:22.145709 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:22.145698 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:23.152270 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:23.152218 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" event={"ID":"a2c213af-bd2a-482f-8365-9838d3ce5389","Type":"ContainerStarted","Data":"a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b"} Sep 10 17:56:23.416652 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:23.416551 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:56:23.417871 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:23.417839 2565 patch_prober.go:28] interesting pod/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Sep 10 17:56:23.417995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:23.417893 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Sep 10 17:56:24.417070 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:24.417035 2565 patch_prober.go:28] interesting pod/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Sep 10 17:56:24.417475 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:24.417086 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Sep 10 17:56:25.417022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:25.416987 2565 patch_prober.go:28] interesting pod/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Sep 10 17:56:25.417202 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:25.417037 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Sep 10 17:56:26.416596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:26.416561 2565 patch_prober.go:28] interesting pod/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Sep 10 17:56:26.416771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:26.416628 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Sep 10 17:56:27.417117 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:27.417080 2565 patch_prober.go:28] interesting pod/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Sep 10 17:56:27.417516 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:27.417145 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Sep 10 17:56:27.981976 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:27.981934 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:56:28.417279 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:28.417184 2565 patch_prober.go:28] interesting pod/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Sep 10 17:56:28.417279 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:28.417263 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Sep 10 17:56:29.421367 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:29.421334 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:56:29.422204 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:29.422184 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:56:29.944056 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:29.944016 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:31.256764 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.256727 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v"] Sep 10 17:56:31.257214 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.256947 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" containerID="cri-o://a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b" gracePeriod=30 Sep 10 17:56:31.405792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.405762 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:31.683992 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.683942 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:31.955780 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.955706 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs"] Sep 10 17:56:31.956169 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.956155 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" Sep 10 17:56:31.956217 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.956170 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" Sep 10 17:56:31.956285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.956274 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" Sep 10 17:56:31.956322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.956288 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" Sep 10 17:56:31.958212 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.958197 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:31.961098 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.961068 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d0df615c-openshift-default-dockercfg-6cv5q\"" Sep 10 17:56:31.969489 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.969468 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs"] Sep 10 17:56:31.973044 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:31.973021 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:32.065995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.065950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-credential-socket\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.066279 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.066257 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-token\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.066417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.066402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-data\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.066523 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.066509 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-envoy\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.066756 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.066742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-certs\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.066861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.066847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-socket\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.066973 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.066960 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-podinfo\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.067080 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.067066 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgrzb\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-kube-api-access-mgrzb\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.067211 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.067196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istiod-ca-cert\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168266 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-credential-socket\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168266 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168270 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-token\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168291 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-data\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-envoy\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168600 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-certs\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168638 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168618 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-socket\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168647 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-data\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168685 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-podinfo\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168767 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168688 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-credential-socket\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168767 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgrzb\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-kube-api-access-mgrzb\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168875 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168773 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istiod-ca-cert\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.168943 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168915 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-certs\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.169001 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.168970 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-socket\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.169335 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.169315 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istiod-ca-cert\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.170745 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.170727 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-envoy\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.171067 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.171045 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-podinfo\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.176011 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.175987 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-token\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.176364 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.176343 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgrzb\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-kube-api-access-mgrzb\") pod \"e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.269647 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.269559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:32.467389 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.467353 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:32.571953 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.571848 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs"] Sep 10 17:56:32.575059 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:56:32.575030 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52fdc0e8_5dd2_4ad8_9428_6ca5e49c8b8d.slice/crio-66f9a2f519a7c79c2de94954b0642130d5cce27a5a3d67d22de1b1853ce3c37e WatchSource:0}: Error finding container 66f9a2f519a7c79c2de94954b0642130d5cce27a5a3d67d22de1b1853ce3c37e: Status 404 returned error can't find the container with id 66f9a2f519a7c79c2de94954b0642130d5cce27a5a3d67d22de1b1853ce3c37e Sep 10 17:56:32.577299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.577261 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:32.577380 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.577333 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:32.577380 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:32.577362 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:33.216357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:33.216324 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" event={"ID":"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d","Type":"ContainerStarted","Data":"ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5"} Sep 10 17:56:33.216524 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:33.216419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" event={"ID":"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d","Type":"ContainerStarted","Data":"66f9a2f519a7c79c2de94954b0642130d5cce27a5a3d67d22de1b1853ce3c37e"} Sep 10 17:56:33.239043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:33.238978 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podStartSLOduration=2.238960971 podStartE2EDuration="2.238960971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:56:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:33.23832943 +0000 UTC m=+922.120054570" watchObservedRunningTime="2026-09-10 17:56:33.238960971 +0000 UTC m=+922.120686113" Sep 10 17:56:33.270504 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:33.270475 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:33.275518 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:33.275497 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:34.221290 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:34.221259 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:34.222154 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:34.222132 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:34.594929 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:34.594823 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:34.686238 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:34.686195 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:35.748488 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:35.748457 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:35.849142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:35.849112 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:36.319344 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.319313 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:36.420370 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.420340 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:36.555323 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.554943 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:56:36.719308 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719279 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c213af-bd2a-482f-8365-9838d3ce5389-istiod-ca-cert\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719317 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mh7w\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-kube-api-access-7mh7w\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719358 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-socket\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719389 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-certs\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719482 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719421 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-credential-socket\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719488 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-token\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719541 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-data\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719577 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-podinfo\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719631 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-envoy\") pod \"a2c213af-bd2a-482f-8365-9838d3ce5389\" (UID: \"a2c213af-bd2a-482f-8365-9838d3ce5389\") " Sep 10 17:56:36.719693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719648 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:36.719693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719664 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:36.720015 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719691 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c213af-bd2a-482f-8365-9838d3ce5389-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:56:36.720015 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719759 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:36.720015 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.719797 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:36.720176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.720127 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.720176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.720146 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c213af-bd2a-482f-8365-9838d3ce5389-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.720176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.720159 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.720176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.720173 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.720430 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.720186 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.721973 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.721947 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:56:36.722050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.721980 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:56:36.722201 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.722176 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:56:36.722319 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.722214 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-kube-api-access-7mh7w" (OuterVolumeSpecName: "kube-api-access-7mh7w") pod "a2c213af-bd2a-482f-8365-9838d3ce5389" (UID: "a2c213af-bd2a-482f-8365-9838d3ce5389"). InnerVolumeSpecName "kube-api-access-7mh7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:56:36.821547 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.821494 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mh7w\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-kube-api-access-7mh7w\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.821547 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.821516 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.821547 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.821525 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:36.821547 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:36.821533 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c213af-bd2a-482f-8365-9838d3ce5389-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:37.240899 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.240858 2565 generic.go:356] "Generic (PLEG): container finished" podID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerID="a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b" exitCode=0 Sep 10 17:56:37.241109 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.240987 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" event={"ID":"a2c213af-bd2a-482f-8365-9838d3ce5389","Type":"ContainerDied","Data":"a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b"} Sep 10 17:56:37.241109 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.241069 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" event={"ID":"a2c213af-bd2a-482f-8365-9838d3ce5389","Type":"ContainerDied","Data":"bff832dc818a9c871026d1b12996a98a6377ea72513ac132a2a797e303f11c0f"} Sep 10 17:56:37.241252 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.241109 2565 scope.go:117] "RemoveContainer" containerID="a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b" Sep 10 17:56:37.241532 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.241340 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v" Sep 10 17:56:37.252445 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.252425 2565 scope.go:117] "RemoveContainer" containerID="9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03" Sep 10 17:56:37.260182 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.260162 2565 scope.go:117] "RemoveContainer" containerID="a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b" Sep 10 17:56:37.260954 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:56:37.260934 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b\": container with ID starting with a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b not found: ID does not exist" containerID="a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b" Sep 10 17:56:37.261037 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.260966 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b"} err="failed to get container status \"a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b\": rpc error: code = NotFound desc = could not find container \"a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b\": container with ID starting with a357dca5ec39460c4ab318954a4e3473bfeee1bac62f4515382a92c60294550b not found: ID does not exist" Sep 10 17:56:37.261037 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.260988 2565 scope.go:117] "RemoveContainer" containerID="9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03" Sep 10 17:56:37.261253 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:56:37.261218 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03\": container with ID starting with 9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03 not found: ID does not exist" containerID="9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03" Sep 10 17:56:37.261309 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.261259 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03"} err="failed to get container status \"9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03\": rpc error: code = NotFound desc = could not find container \"9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03\": container with ID starting with 9b3b24fd8565e5370272eb1912a72fd45bdc5fe8b0cf817af935c6e62ed8bd03 not found: ID does not exist" Sep 10 17:56:37.266492 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.266470 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v"] Sep 10 17:56:37.272629 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.272598 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9584e914-openshift-default-ccf695fb6-zv79v"] Sep 10 17:56:37.623086 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:37.623009 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" path="/var/lib/kubelet/pods/a2c213af-bd2a-482f-8365-9838d3ce5389/volumes" Sep 10 17:56:38.599337 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:38.599308 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:39.510200 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:39.510173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:44.803009 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:44.802975 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:45.028315 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:45.028263 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:50.263413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:50.263364 2565 patch_prober.go:28] interesting pod/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 17:56:50.263990 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:50.263440 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 17:56:52.327760 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:52.327668 2565 generic.go:356] "Generic (PLEG): container finished" podID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerID="ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5" exitCode=0 Sep 10 17:56:52.328142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:52.327751 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" event={"ID":"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d","Type":"ContainerDied","Data":"ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5"} Sep 10 17:56:52.328142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:52.328122 2565 scope.go:117] "RemoveContainer" containerID="ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5" Sep 10 17:56:52.329199 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:52.329157 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:52.329326 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:52.329278 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:52.329392 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:52.329332 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:56:53.334943 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:53.334909 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" event={"ID":"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d","Type":"ContainerStarted","Data":"520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49"} Sep 10 17:56:54.270772 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:54.270729 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:54.270982 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:54.270865 2565 patch_prober.go:28] interesting pod/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 10 17:56:54.270982 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:54.270913 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 10 17:56:55.270074 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.270038 2565 patch_prober.go:28] interesting pod/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 10 17:56:55.270708 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.270673 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 10 17:56:55.597123 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.597037 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:55.648784 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.648751 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2"] Sep 10 17:56:55.649024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.648995 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" podUID="d9dd25c5-c6c7-4275-b41e-c58399bf20e3" containerName="payload-processing" containerID="cri-o://8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024" gracePeriod=30 Sep 10 17:56:55.691077 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.691043 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:55.773707 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.773671 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:55.859422 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.855728 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:55.913542 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:55.913483 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:56.005099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.004804 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:56:56.115196 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.115108 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vclnv\" (UniqueName: \"kubernetes.io/projected/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-kube-api-access-vclnv\") pod \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " Sep 10 17:56:56.115376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.115251 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-plugins-config-volume\") pod \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\" (UID: \"d9dd25c5-c6c7-4275-b41e-c58399bf20e3\") " Sep 10 17:56:56.115594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.115564 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9dd25c5-c6c7-4275-b41e-c58399bf20e3" (UID: "d9dd25c5-c6c7-4275-b41e-c58399bf20e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:56:56.117195 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.117170 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-kube-api-access-vclnv" (OuterVolumeSpecName: "kube-api-access-vclnv") pod "d9dd25c5-c6c7-4275-b41e-c58399bf20e3" (UID: "d9dd25c5-c6c7-4275-b41e-c58399bf20e3"). InnerVolumeSpecName "kube-api-access-vclnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:56:56.187130 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.187096 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:56.216010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.215975 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vclnv\" (UniqueName: \"kubernetes.io/projected/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-kube-api-access-vclnv\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:56.216010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.216007 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dd25c5-c6c7-4275-b41e-c58399bf20e3-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:56:56.270095 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.270046 2565 patch_prober.go:28] interesting pod/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 10 17:56:56.270598 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.270119 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 10 17:56:56.366299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.363303 2565 generic.go:356] "Generic (PLEG): container finished" podID="d9dd25c5-c6c7-4275-b41e-c58399bf20e3" containerID="8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024" exitCode=0 Sep 10 17:56:56.366299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.363450 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" event={"ID":"d9dd25c5-c6c7-4275-b41e-c58399bf20e3","Type":"ContainerDied","Data":"8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024"} Sep 10 17:56:56.366299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.363504 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" event={"ID":"d9dd25c5-c6c7-4275-b41e-c58399bf20e3","Type":"ContainerDied","Data":"6887edc9c4f693b19fd2cd1c0841a596fd535f31bdfa2d7ada0acab124b045ce"} Sep 10 17:56:56.366299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.363535 2565 scope.go:117] "RemoveContainer" containerID="8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024" Sep 10 17:56:56.366299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.363702 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2" Sep 10 17:56:56.383899 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.383880 2565 scope.go:117] "RemoveContainer" containerID="8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024" Sep 10 17:56:56.384179 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:56:56.384156 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024\": container with ID starting with 8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024 not found: ID does not exist" containerID="8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024" Sep 10 17:56:56.384261 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.384191 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024"} err="failed to get container status \"8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024\": rpc error: code = NotFound desc = could not find container \"8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024\": container with ID starting with 8a2d370e053a79c34773fe1ed73fa81c18c16b8c80321df771f6f74f45e28024 not found: ID does not exist" Sep 10 17:56:56.421289 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.421259 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2"] Sep 10 17:56:56.438207 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:56.438179 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bcdd7c-6b44bb9947-dj7b2"] Sep 10 17:56:57.270351 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:57.270314 2565 patch_prober.go:28] interesting pod/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 10 17:56:57.270941 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:57.270902 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 10 17:56:57.622855 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:57.622771 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9dd25c5-c6c7-4275-b41e-c58399bf20e3" path="/var/lib/kubelet/pods/d9dd25c5-c6c7-4275-b41e-c58399bf20e3/volumes" Sep 10 17:56:58.069902 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:58.069872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:58.270158 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:58.270119 2565 patch_prober.go:28] interesting pod/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 10 17:56:58.270491 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:58.270463 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 10 17:56:59.273766 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:59.273738 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:59.274107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:59.274026 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:56:59.274723 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:56:59.274706 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:57:02.161751 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:02.161712 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:02.262635 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:02.262601 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:02.384872 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:02.384838 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk"] Sep 10 17:57:02.385085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:02.385061 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" containerID="cri-o://c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564" gracePeriod=30 Sep 10 17:57:02.406413 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:02.406363 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:02.816676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:02.816641 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:03.134260 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134145 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x"] Sep 10 17:57:03.134776 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134756 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" Sep 10 17:57:03.134840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134777 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" Sep 10 17:57:03.134840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134786 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" Sep 10 17:57:03.134840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134792 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a704794-30a3-4193-9f93-b110caee61f0" containerName="istio-proxy" Sep 10 17:57:03.134840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134831 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" Sep 10 17:57:03.134840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134839 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" Sep 10 17:57:03.134993 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134849 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9dd25c5-c6c7-4275-b41e-c58399bf20e3" containerName="payload-processing" Sep 10 17:57:03.134993 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134854 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9dd25c5-c6c7-4275-b41e-c58399bf20e3" containerName="payload-processing" Sep 10 17:57:03.134993 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134917 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" Sep 10 17:57:03.134993 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.134928 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9dd25c5-c6c7-4275-b41e-c58399bf20e3" containerName="payload-processing" Sep 10 17:57:03.138013 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.137996 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.141273 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.141255 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-c4540c-openshift-default-dockercfg-kmvsc\"" Sep 10 17:57:03.152958 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.152932 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x"] Sep 10 17:57:03.163586 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.163565 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:03.291368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291205 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-envoy\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-istio-token\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291328 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-socket\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291368 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9f877a-9860-4d68-b422-565c350d3697-istio-podinfo\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-credential-socket\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291410 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-data\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291447 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9f877a-9860-4d68-b422-565c350d3697-istiod-ca-cert\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291503 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-certs\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.291728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.291564 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzvbk\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-kube-api-access-xzvbk\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.374021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.373988 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:03.393561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.393479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-envoy\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.393822 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.393800 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-istio-token\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.393966 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.393951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-socket\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394069 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394056 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9f877a-9860-4d68-b422-565c350d3697-istio-podinfo\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394171 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394157 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-credential-socket\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394295 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394281 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-data\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394407 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394394 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9f877a-9860-4d68-b422-565c350d3697-istiod-ca-cert\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394541 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-certs\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394684 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394668 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzvbk\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-kube-api-access-xzvbk\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.394853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394830 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-data\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.395082 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394560 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-credential-socket\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.395183 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.395158 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9f877a-9860-4d68-b422-565c350d3697-istiod-ca-cert\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.395275 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.394528 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-socket\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.395537 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.395514 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-certs\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.397606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.397570 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9f877a-9860-4d68-b422-565c350d3697-istio-podinfo\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.398182 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.398156 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-envoy\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.407378 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.407350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-istio-token\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.408108 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.408004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzvbk\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-kube-api-access-xzvbk\") pod \"e2e-explicit-c4540c-openshift-default-766b794968-nsf5x\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.448786 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.448755 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:03.751458 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.751435 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x"] Sep 10 17:57:03.753162 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:57:03.753134 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9f877a_9860_4d68_b422_565c350d3697.slice/crio-8bd135401d66ad8b5b3ee1b876beed885c36863df96e712f94dc4a134fac09d4 WatchSource:0}: Error finding container 8bd135401d66ad8b5b3ee1b876beed885c36863df96e712f94dc4a134fac09d4: Status 404 returned error can't find the container with id 8bd135401d66ad8b5b3ee1b876beed885c36863df96e712f94dc4a134fac09d4 Sep 10 17:57:03.755370 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.755337 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:57:03.755451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.755401 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:57:03.755451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:03.755434 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:57:04.421605 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.421573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" event={"ID":"dd9f877a-9860-4d68-b422-565c350d3697","Type":"ContainerStarted","Data":"252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a"} Sep 10 17:57:04.421987 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.421635 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" event={"ID":"dd9f877a-9860-4d68-b422-565c350d3697","Type":"ContainerStarted","Data":"8bd135401d66ad8b5b3ee1b876beed885c36863df96e712f94dc4a134fac09d4"} Sep 10 17:57:04.445287 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.445240 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podStartSLOduration=1.445212611 podStartE2EDuration="1.445212611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:57:04.442742969 +0000 UTC m=+953.324468112" watchObservedRunningTime="2026-09-10 17:57:04.445212611 +0000 UTC m=+953.326937749" Sep 10 17:57:04.449880 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.449857 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:04.454520 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.454499 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:04.646949 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.646917 2565 patch_prober.go:28] interesting pod/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 17:57:04.647102 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:04.646984 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:57:05.426963 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:05.426933 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:05.427917 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:05.427895 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:06.333411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.333365 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:06.420178 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.420131 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:06.507670 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.507638 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:06.606824 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.606734 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:06.711514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.711476 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:06.770804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.770770 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:06.924155 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.924123 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs"] Sep 10 17:57:06.924455 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.924429 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" containerID="cri-o://520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49" gracePeriod=30 Sep 10 17:57:06.936950 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:06.936924 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:07.549054 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.549021 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:07.642480 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.642454 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:57:07.642603 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.642586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:07.737332 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737295 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-socket\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737348 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-data\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737374 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff6zp\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-kube-api-access-ff6zp\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737429 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-certs\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737463 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cd9904-941c-4737-baa8-f9d71103d27a-istiod-ca-cert\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737488 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-envoy\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737513 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-istio-token\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737581 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cd9904-941c-4737-baa8-f9d71103d27a-istio-podinfo\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737599 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-data" (OuterVolumeSpecName: "istio-data") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:07.737861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737615 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:07.737861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737628 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-credential-socket\") pod \"78cd9904-941c-4737-baa8-f9d71103d27a\" (UID: \"78cd9904-941c-4737-baa8-f9d71103d27a\") " Sep 10 17:57:07.737861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737723 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:07.737861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737824 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78cd9904-941c-4737-baa8-f9d71103d27a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:57:07.738175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.737952 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:07.738321 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.738301 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.738396 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.738325 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.738396 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.738338 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.738396 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.738352 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.738396 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.738365 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cd9904-941c-4737-baa8-f9d71103d27a-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.739884 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.739859 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78cd9904-941c-4737-baa8-f9d71103d27a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:57:07.740017 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.739888 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-istio-token" (OuterVolumeSpecName: "istio-token") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:57:07.740245 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.740191 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:07.740356 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.740317 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-kube-api-access-ff6zp" (OuterVolumeSpecName: "kube-api-access-ff6zp") pod "78cd9904-941c-4737-baa8-f9d71103d27a" (UID: "78cd9904-941c-4737-baa8-f9d71103d27a"). InnerVolumeSpecName "kube-api-access-ff6zp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:57:07.743083 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.743062 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:07.839412 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.839387 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff6zp\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-kube-api-access-ff6zp\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.839412 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.839409 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cd9904-941c-4737-baa8-f9d71103d27a-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.839534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.839418 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cd9904-941c-4737-baa8-f9d71103d27a-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.839534 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.839426 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cd9904-941c-4737-baa8-f9d71103d27a-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:07.843992 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:07.843971 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:08.132066 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.131991 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:08.250918 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.250886 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:08.445079 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.445045 2565 generic.go:356] "Generic (PLEG): container finished" podID="78cd9904-941c-4737-baa8-f9d71103d27a" containerID="c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564" exitCode=0 Sep 10 17:57:08.445258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.445079 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" event={"ID":"78cd9904-941c-4737-baa8-f9d71103d27a","Type":"ContainerDied","Data":"c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564"} Sep 10 17:57:08.445258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.445134 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" event={"ID":"78cd9904-941c-4737-baa8-f9d71103d27a","Type":"ContainerDied","Data":"5320135b2da715d75d0542b0479db0f5b1d81669efeccef2d86364d54eb331c3"} Sep 10 17:57:08.445258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.445091 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk" Sep 10 17:57:08.445258 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.445192 2565 scope.go:117] "RemoveContainer" containerID="c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564" Sep 10 17:57:08.455485 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.455471 2565 scope.go:117] "RemoveContainer" containerID="0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49" Sep 10 17:57:08.463509 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.463491 2565 scope.go:117] "RemoveContainer" containerID="c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564" Sep 10 17:57:08.463771 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:57:08.463751 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564\": container with ID starting with c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564 not found: ID does not exist" containerID="c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564" Sep 10 17:57:08.463842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.463782 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564"} err="failed to get container status \"c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564\": rpc error: code = NotFound desc = could not find container \"c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564\": container with ID starting with c0fd1a459ec59f8ff3b8c7f8a1dc660b33730bfda6f38212ea60b58ffa6cf564 not found: ID does not exist" Sep 10 17:57:08.463842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.463805 2565 scope.go:117] "RemoveContainer" containerID="0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49" Sep 10 17:57:08.464079 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:57:08.464061 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49\": container with ID starting with 0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49 not found: ID does not exist" containerID="0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49" Sep 10 17:57:08.464121 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.464085 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49"} err="failed to get container status \"0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49\": rpc error: code = NotFound desc = could not find container \"0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49\": container with ID starting with 0bf69d82815d128382e916ed93ca6bda91c8b2775f976aba1714cc6dfc5b3e49 not found: ID does not exist" Sep 10 17:57:08.468218 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.468194 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk"] Sep 10 17:57:08.472476 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:08.472458 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-bcdd7c-openshift-default-8f59f9f64-ljvtk"] Sep 10 17:57:09.160520 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:09.160483 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:09.622999 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:09.622969 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" path="/var/lib/kubelet/pods/78cd9904-941c-4737-baa8-f9d71103d27a/volumes" Sep 10 17:57:12.169647 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.169625 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:57:12.286474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286401 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgrzb\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-kube-api-access-mgrzb\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286446 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-data\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286469 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-certs\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286504 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istiod-ca-cert\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286562 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-envoy\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286589 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-credential-socket\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286614 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-socket\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.286742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286663 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:12.286742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286711 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-podinfo\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.287050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286751 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-token\") pod \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\" (UID: \"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d\") " Sep 10 17:57:12.287050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286831 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:12.287050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286850 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:12.287050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286933 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:12.287050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.286961 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:57:12.287314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.287165 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.287314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.287186 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.287314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.287201 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.287314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.287214 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.287314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.287248 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.288717 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.288694 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:57:12.288835 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.288758 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-kube-api-access-mgrzb" (OuterVolumeSpecName: "kube-api-access-mgrzb") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "kube-api-access-mgrzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:57:12.289184 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.289167 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:57:12.289277 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.289169 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" (UID: "52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:57:12.388187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.388164 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.388187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.388186 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgrzb\" (UniqueName: \"kubernetes.io/projected/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-kube-api-access-mgrzb\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.388356 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.388195 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.388356 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.388204 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:12.471357 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.471330 2565 generic.go:356] "Generic (PLEG): container finished" podID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerID="520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49" exitCode=0 Sep 10 17:57:12.471474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.471379 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" Sep 10 17:57:12.471474 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.471423 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" event={"ID":"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d","Type":"ContainerDied","Data":"520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49"} Sep 10 17:57:12.471545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.471473 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs" event={"ID":"52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d","Type":"ContainerDied","Data":"66f9a2f519a7c79c2de94954b0642130d5cce27a5a3d67d22de1b1853ce3c37e"} Sep 10 17:57:12.471545 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.471497 2565 scope.go:117] "RemoveContainer" containerID="520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49" Sep 10 17:57:12.480476 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.480459 2565 scope.go:117] "RemoveContainer" containerID="ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5" Sep 10 17:57:12.490412 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.489783 2565 scope.go:117] "RemoveContainer" containerID="520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49" Sep 10 17:57:12.490412 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:57:12.490402 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49\": container with ID starting with 520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49 not found: ID does not exist" containerID="520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49" Sep 10 17:57:12.490542 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.490435 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49"} err="failed to get container status \"520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49\": rpc error: code = NotFound desc = could not find container \"520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49\": container with ID starting with 520e165d444ef189659669f441aa253bdcb0aa35db93c72aab21d3a736030e49 not found: ID does not exist" Sep 10 17:57:12.490542 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.490459 2565 scope.go:117] "RemoveContainer" containerID="ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5" Sep 10 17:57:12.491139 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:57:12.491111 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5\": container with ID starting with ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5 not found: ID does not exist" containerID="ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5" Sep 10 17:57:12.491251 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.491146 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5"} err="failed to get container status \"ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5\": rpc error: code = NotFound desc = could not find container \"ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5\": container with ID starting with ac7f3865c11fafc6c280fe8b78c1d76a0b3a2d39eb6ba531fd11a065f9cab1b5 not found: ID does not exist" Sep 10 17:57:12.493463 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.493444 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs"] Sep 10 17:57:12.497680 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:12.497659 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d0df615c-openshift-default-7d8ddd6cf6-lgpvs"] Sep 10 17:57:13.622970 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:13.622938 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" path="/var/lib/kubelet/pods/52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d/volumes" Sep 10 17:57:15.205345 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:15.205305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:15.263641 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:15.263605 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:15.346246 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:15.346184 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:15.416945 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:15.416916 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:15.479591 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:15.479513 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:15.878494 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:15.878414 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:18.076449 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:18.076413 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:28.556900 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:28.556802 2565 generic.go:356] "Generic (PLEG): container finished" podID="dd9f877a-9860-4d68-b422-565c350d3697" containerID="252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a" exitCode=0 Sep 10 17:57:28.556900 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:28.556877 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" event={"ID":"dd9f877a-9860-4d68-b422-565c350d3697","Type":"ContainerDied","Data":"252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a"} Sep 10 17:57:28.557388 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:28.557372 2565 scope.go:117] "RemoveContainer" containerID="252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a" Sep 10 17:57:28.558441 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:28.558415 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:57:28.558502 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:28.558487 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:57:28.558538 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:28.558516 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:57:29.568552 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:29.568491 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" event={"ID":"dd9f877a-9860-4d68-b422-565c350d3697","Type":"ContainerStarted","Data":"7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425"} Sep 10 17:57:30.449411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:30.449371 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:30.449670 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:30.449574 2565 patch_prober.go:28] interesting pod/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Sep 10 17:57:30.449670 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:30.449622 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Sep 10 17:57:31.275000 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:31.274961 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:31.449950 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:31.449913 2565 patch_prober.go:28] interesting pod/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Sep 10 17:57:31.450126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:31.449972 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Sep 10 17:57:32.449766 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:32.449721 2565 patch_prober.go:28] interesting pod/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Sep 10 17:57:32.450137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:32.449784 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Sep 10 17:57:33.449884 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:33.449846 2565 patch_prober.go:28] interesting pod/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Sep 10 17:57:33.450395 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:33.449908 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Sep 10 17:57:33.717833 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:33.717760 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:34.450015 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:34.449980 2565 patch_prober.go:28] interesting pod/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Sep 10 17:57:34.450443 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:34.450033 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Sep 10 17:57:35.427526 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:35.427495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:35.453385 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:35.453360 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:35.606141 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:35.606114 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:57:41.328207 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:41.328172 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:41.787933 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:41.787904 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:47.758535 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.758478 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:47.864771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.864732 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj"] Sep 10 17:57:47.865508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865483 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" Sep 10 17:57:47.865508 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865506 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865529 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865537 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865556 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865566 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865577 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865584 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865691 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" Sep 10 17:57:47.865705 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865701 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" Sep 10 17:57:47.866091 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865713 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cd9904-941c-4737-baa8-f9d71103d27a" containerName="istio-proxy" Sep 10 17:57:47.866091 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.865723 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c213af-bd2a-482f-8365-9838d3ce5389" containerName="istio-proxy" Sep 10 17:57:47.868085 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.868064 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:47.870836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.870815 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 17:57:47.871043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.871015 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 17:57:47.877138 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.877031 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj"] Sep 10 17:57:47.943526 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.943498 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:47.953805 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:47.953781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2608f459-8377-4c76-b646-679c3bc04e00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj\" (UID: \"2608f459-8377-4c76-b646-679c3bc04e00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:48.047617 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.047557 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:48.054781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.054760 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2608f459-8377-4c76-b646-679c3bc04e00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj\" (UID: \"2608f459-8377-4c76-b646-679c3bc04e00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:48.055380 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.055364 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2608f459-8377-4c76-b646-679c3bc04e00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj\" (UID: \"2608f459-8377-4c76-b646-679c3bc04e00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:48.147126 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.147096 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:48.181576 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.181554 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:48.449271 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.449248 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj"] Sep 10 17:57:48.451036 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:57:48.451011 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2608f459_8377_4c76_b646_679c3bc04e00.slice/crio-b3fcf5e6539e1d10b1345fd90a825a812e96038905b80ce71d7b7646d7ed304b WatchSource:0}: Error finding container b3fcf5e6539e1d10b1345fd90a825a812e96038905b80ce71d7b7646d7ed304b: Status 404 returned error can't find the container with id b3fcf5e6539e1d10b1345fd90a825a812e96038905b80ce71d7b7646d7ed304b Sep 10 17:57:48.453068 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.453052 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:57:48.667804 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:48.667770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" event={"ID":"2608f459-8377-4c76-b646-679c3bc04e00","Type":"ContainerStarted","Data":"b3fcf5e6539e1d10b1345fd90a825a812e96038905b80ce71d7b7646d7ed304b"} Sep 10 17:57:51.688375 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:51.688337 2565 generic.go:356] "Generic (PLEG): container finished" podID="2608f459-8377-4c76-b646-679c3bc04e00" containerID="f486aa104bf52aa0899b25aaffaea4692300f71175928959627ae3b0f3ddf2da" exitCode=0 Sep 10 17:57:51.688785 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:51.688420 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" event={"ID":"2608f459-8377-4c76-b646-679c3bc04e00","Type":"ContainerDied","Data":"f486aa104bf52aa0899b25aaffaea4692300f71175928959627ae3b0f3ddf2da"} Sep 10 17:57:52.826772 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:52.826752 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:52.997616 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:52.997523 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2608f459-8377-4c76-b646-679c3bc04e00-maas-api-service-ca\") pod \"2608f459-8377-4c76-b646-679c3bc04e00\" (UID: \"2608f459-8377-4c76-b646-679c3bc04e00\") " Sep 10 17:57:52.997831 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:52.997809 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2608f459-8377-4c76-b646-679c3bc04e00-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2608f459-8377-4c76-b646-679c3bc04e00" (UID: "2608f459-8377-4c76-b646-679c3bc04e00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:57:53.098549 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:53.098521 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2608f459-8377-4c76-b646-679c3bc04e00-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:57:53.699503 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:53.699473 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" event={"ID":"2608f459-8377-4c76-b646-679c3bc04e00","Type":"ContainerDied","Data":"b3fcf5e6539e1d10b1345fd90a825a812e96038905b80ce71d7b7646d7ed304b"} Sep 10 17:57:53.699636 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:53.699507 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj" Sep 10 17:57:53.699636 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:53.699517 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3fcf5e6539e1d10b1345fd90a825a812e96038905b80ce71d7b7646d7ed304b" Sep 10 17:57:58.319781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.319741 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:58.377309 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.377278 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:58.438846 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.438803 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:58.497849 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.497802 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj"] Sep 10 17:57:58.499682 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.499654 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:58.500755 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.500733 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c4540c-nm6jj"] Sep 10 17:57:58.554953 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.554927 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:58.852248 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:58.852186 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:57:59.622838 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:57:59.622807 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2608f459-8377-4c76-b646-679c3bc04e00" path="/var/lib/kubelet/pods/2608f459-8377-4c76-b646-679c3bc04e00/volumes" Sep 10 17:58:04.005332 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:04.005299 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:04.879176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:04.879146 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:04.931930 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:04.931897 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:05.119981 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.119939 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x"] Sep 10 17:58:05.120452 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.120170 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" containerID="cri-o://7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425" gracePeriod=30 Sep 10 17:58:05.133074 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.133008 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:05.595374 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.595336 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:05.607021 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.606995 2565 patch_prober.go:28] interesting pod/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 17:58:05.607151 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.607069 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:58:05.861426 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.861340 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2"] Sep 10 17:58:05.861759 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.861743 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2608f459-8377-4c76-b646-679c3bc04e00" containerName="revoke-keys" Sep 10 17:58:05.861759 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.861760 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2608f459-8377-4c76-b646-679c3bc04e00" containerName="revoke-keys" Sep 10 17:58:05.861894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.861834 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2608f459-8377-4c76-b646-679c3bc04e00" containerName="revoke-keys" Sep 10 17:58:05.861894 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.861843 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52fdc0e8-5dd2-4ad8-9428-6ca5e49c8b8d" containerName="istio-proxy" Sep 10 17:58:05.864525 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.864503 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.867253 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.867217 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-c3fa7f-openshift-default-dockercfg-fkz8b\"" Sep 10 17:58:05.876446 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.876422 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2"] Sep 10 17:58:05.882775 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.882755 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:05.917607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917581 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-data\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.917724 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-credential-socket\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.917724 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-envoy\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.917812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917735 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-podinfo\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.917812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dadf206-90e5-40f8-aaf1-819dede87a1b-istiod-ca-cert\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.917812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917789 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-certs\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.917812 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-token\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.918022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917876 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-socket\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:05.918022 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:05.917898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl7qb\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-kube-api-access-gl7qb\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019371 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-credential-socket\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-envoy\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019472 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-podinfo\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dadf206-90e5-40f8-aaf1-819dede87a1b-istiod-ca-cert\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019525 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-certs\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-token\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019607 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019586 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-socket\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019951 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl7qb\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-kube-api-access-gl7qb\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019951 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-data\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.019951 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019842 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-credential-socket\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.020132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.019949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-socket\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.020132 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.020030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-data\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.020269 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.020218 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-certs\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.020319 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.020267 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dadf206-90e5-40f8-aaf1-819dede87a1b-istiod-ca-cert\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.021832 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.021810 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-envoy\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.021934 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.021866 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-podinfo\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.027204 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.027179 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-token\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.027410 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.027391 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl7qb\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-kube-api-access-gl7qb\") pod \"e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.132209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.132148 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:06.175270 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.175245 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:06.455673 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.455649 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2"] Sep 10 17:58:06.457742 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:58:06.457711 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dadf206_90e5_40f8_aaf1_819dede87a1b.slice/crio-9b229925335728fe8cff14787e10d51553d524ad53d444cc3b7a94b1cc73df2b WatchSource:0}: Error finding container 9b229925335728fe8cff14787e10d51553d524ad53d444cc3b7a94b1cc73df2b: Status 404 returned error can't find the container with id 9b229925335728fe8cff14787e10d51553d524ad53d444cc3b7a94b1cc73df2b Sep 10 17:58:06.459882 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.459848 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:58:06.460004 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.459914 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:58:06.460004 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.459944 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:58:06.765764 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.765674 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" event={"ID":"5dadf206-90e5-40f8-aaf1-819dede87a1b","Type":"ContainerStarted","Data":"d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04"} Sep 10 17:58:06.765764 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.765742 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" event={"ID":"5dadf206-90e5-40f8-aaf1-819dede87a1b","Type":"ContainerStarted","Data":"9b229925335728fe8cff14787e10d51553d524ad53d444cc3b7a94b1cc73df2b"} Sep 10 17:58:06.789842 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:06.789798 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" podStartSLOduration=1.7897827450000001 podStartE2EDuration="1.789782745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:06.786728954 +0000 UTC m=+1015.668454095" watchObservedRunningTime="2026-09-10 17:58:06.789782745 +0000 UTC m=+1015.671507885" Sep 10 17:58:07.059115 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.059040 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:07.158019 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.157986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:07.176032 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.176012 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:07.180719 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.180699 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:07.270369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.270346 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:07.771443 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.771414 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:07.772427 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:07.772404 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:10.188523 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.188494 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:10.253619 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.253593 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:10.330596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.330558 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:10.374382 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.374363 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:58:10.400401 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.400340 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:10.462781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.462754 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9f877a-9860-4d68-b422-565c350d3697-istio-podinfo\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.462919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.462797 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-istio-token\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.462919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.462862 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-credential-socket\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.462919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.462881 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9f877a-9860-4d68-b422-565c350d3697-istiod-ca-cert\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.463052 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.462947 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-certs\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.463052 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.462992 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-envoy\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.463052 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463018 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzvbk\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-kube-api-access-xzvbk\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.463203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463061 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-data\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.463203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463100 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-socket\") pod \"dd9f877a-9860-4d68-b422-565c350d3697\" (UID: \"dd9f877a-9860-4d68-b422-565c350d3697\") " Sep 10 17:58:10.463203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463127 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:10.463417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463289 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:10.463417 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463389 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:10.463529 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463478 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:10.463529 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463489 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd9f877a-9860-4d68-b422-565c350d3697-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:58:10.463645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463570 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.463645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463583 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.463645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463595 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.463645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463609 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9f877a-9860-4d68-b422-565c350d3697-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.463645 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.463622 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.465110 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.465088 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd9f877a-9860-4d68-b422-565c350d3697-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:58:10.465218 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.465108 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:10.465218 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.465166 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:58:10.465411 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.465393 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-kube-api-access-xzvbk" (OuterVolumeSpecName: "kube-api-access-xzvbk") pod "dd9f877a-9860-4d68-b422-565c350d3697" (UID: "dd9f877a-9860-4d68-b422-565c350d3697"). InnerVolumeSpecName "kube-api-access-xzvbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:58:10.564781 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.564755 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9f877a-9860-4d68-b422-565c350d3697-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.564886 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.564785 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzvbk\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-kube-api-access-xzvbk\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.564886 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.564800 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9f877a-9860-4d68-b422-565c350d3697-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.564886 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.564813 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9f877a-9860-4d68-b422-565c350d3697-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:10.789328 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.789296 2565 generic.go:356] "Generic (PLEG): container finished" podID="dd9f877a-9860-4d68-b422-565c350d3697" containerID="7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425" exitCode=0 Sep 10 17:58:10.789487 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.789340 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" Sep 10 17:58:10.789487 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.789382 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" event={"ID":"dd9f877a-9860-4d68-b422-565c350d3697","Type":"ContainerDied","Data":"7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425"} Sep 10 17:58:10.789487 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.789443 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x" event={"ID":"dd9f877a-9860-4d68-b422-565c350d3697","Type":"ContainerDied","Data":"8bd135401d66ad8b5b3ee1b876beed885c36863df96e712f94dc4a134fac09d4"} Sep 10 17:58:10.789487 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.789470 2565 scope.go:117] "RemoveContainer" containerID="7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425" Sep 10 17:58:10.798828 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.798813 2565 scope.go:117] "RemoveContainer" containerID="252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a" Sep 10 17:58:10.806880 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.806865 2565 scope.go:117] "RemoveContainer" containerID="7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425" Sep 10 17:58:10.807145 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:58:10.807126 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425\": container with ID starting with 7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425 not found: ID does not exist" containerID="7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425" Sep 10 17:58:10.807219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.807155 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425"} err="failed to get container status \"7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425\": rpc error: code = NotFound desc = could not find container \"7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425\": container with ID starting with 7c9ae4ee76bfe9c9adfa817f3477041ab9f5a8557edbf168382c0f6f144d3425 not found: ID does not exist" Sep 10 17:58:10.807219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.807177 2565 scope.go:117] "RemoveContainer" containerID="252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a" Sep 10 17:58:10.807460 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:58:10.807441 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a\": container with ID starting with 252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a not found: ID does not exist" containerID="252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a" Sep 10 17:58:10.807503 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.807466 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a"} err="failed to get container status \"252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a\": rpc error: code = NotFound desc = could not find container \"252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a\": container with ID starting with 252ffed483cea7d74e88d919b257627b98fabe9caffbeef755bd173427eeb12a not found: ID does not exist" Sep 10 17:58:10.812370 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.812348 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x"] Sep 10 17:58:10.816219 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:10.816196 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-c4540c-openshift-default-766b794968-nsf5x"] Sep 10 17:58:11.624045 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:11.624013 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd9f877a-9860-4d68-b422-565c350d3697" path="/var/lib/kubelet/pods/dd9f877a-9860-4d68-b422-565c350d3697/volumes" Sep 10 17:58:20.875999 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:20.875967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:21.666995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:21.666957 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:22.024785 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:22.024752 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:27.429409 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.429375 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:27.522451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.522420 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2"] Sep 10 17:58:27.522662 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.522637 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" podUID="5dadf206-90e5-40f8-aaf1-819dede87a1b" containerName="istio-proxy" containerID="cri-o://d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04" gracePeriod=30 Sep 10 17:58:27.534530 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.534504 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:27.755034 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.754956 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:27.864342 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.864305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:27.964570 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:27.964541 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:28.067621 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:28.067529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:28.175054 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:28.175020 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:28.286050 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:28.286016 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:28.857835 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:28.857804 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:28.957549 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:28.957518 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:29.058142 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:29.058117 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:30.550713 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:30.550678 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:32.578527 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.578495 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:32.637063 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.637028 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:32.779030 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.779007 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:32.866546 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866468 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-token\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.866546 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866511 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dadf206-90e5-40f8-aaf1-819dede87a1b-istiod-ca-cert\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.866771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866561 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-envoy\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.866771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866730 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-credential-socket\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.866771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866768 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl7qb\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-kube-api-access-gl7qb\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.866935 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866809 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-certs\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.866935 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866923 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-data\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.867044 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866948 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-socket\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.867044 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866968 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:32.867044 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.866974 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dadf206-90e5-40f8-aaf1-819dede87a1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:58:32.867044 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867004 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-podinfo\") pod \"5dadf206-90e5-40f8-aaf1-819dede87a1b\" (UID: \"5dadf206-90e5-40f8-aaf1-819dede87a1b\") " Sep 10 17:58:32.867254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867155 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:32.867254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867175 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:32.867254 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867187 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:32.867451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867412 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.867451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867438 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.867451 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867452 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.867681 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867463 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.867681 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.867471 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dadf206-90e5-40f8-aaf1-819dede87a1b-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.868914 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.868886 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-kube-api-access-gl7qb" (OuterVolumeSpecName: "kube-api-access-gl7qb") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "kube-api-access-gl7qb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:58:32.869023 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.869004 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:58:32.869160 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.869140 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:58:32.869280 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.869154 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "5dadf206-90e5-40f8-aaf1-819dede87a1b" (UID: "5dadf206-90e5-40f8-aaf1-819dede87a1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:58:32.888336 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.888303 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:32.901987 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.901962 2565 generic.go:356] "Generic (PLEG): container finished" podID="5dadf206-90e5-40f8-aaf1-819dede87a1b" containerID="d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04" exitCode=0 Sep 10 17:58:32.902063 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.902007 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" Sep 10 17:58:32.902063 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.902043 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" event={"ID":"5dadf206-90e5-40f8-aaf1-819dede87a1b","Type":"ContainerDied","Data":"d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04"} Sep 10 17:58:32.902145 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.902117 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2" event={"ID":"5dadf206-90e5-40f8-aaf1-819dede87a1b","Type":"ContainerDied","Data":"9b229925335728fe8cff14787e10d51553d524ad53d444cc3b7a94b1cc73df2b"} Sep 10 17:58:32.902145 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.902140 2565 scope.go:117] "RemoveContainer" containerID="d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04" Sep 10 17:58:32.911036 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.911018 2565 scope.go:117] "RemoveContainer" containerID="d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04" Sep 10 17:58:32.911334 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:58:32.911309 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04\": container with ID starting with d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04 not found: ID does not exist" containerID="d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04" Sep 10 17:58:32.911402 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.911342 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04"} err="failed to get container status \"d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04\": rpc error: code = NotFound desc = could not find container \"d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04\": container with ID starting with d85cf1fdd9035fb5a28dbb606bc871ef1eb6af19e15e532197ddfd840c57dd04 not found: ID does not exist" Sep 10 17:58:32.924008 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.923965 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2"] Sep 10 17:58:32.929276 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.929252 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-c3fa7f-openshift-default-f5fdcb76d-9r2s2"] Sep 10 17:58:32.968319 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.968293 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl7qb\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-kube-api-access-gl7qb\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.968319 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.968316 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.968461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.968327 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:32.968461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:32.968335 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dadf206-90e5-40f8-aaf1-819dede87a1b-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:33.623185 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:33.623153 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dadf206-90e5-40f8-aaf1-819dede87a1b" path="/var/lib/kubelet/pods/5dadf206-90e5-40f8-aaf1-819dede87a1b/volumes" Sep 10 17:58:36.405869 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:36.405835 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:36.459347 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:36.459305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:37.089192 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:37.089150 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:39.552307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:39.552267 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:40.630401 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.630365 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:40.747206 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747177 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7"] Sep 10 17:58:40.747596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747582 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" Sep 10 17:58:40.747642 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747599 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" Sep 10 17:58:40.747642 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747624 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" Sep 10 17:58:40.747642 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747630 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" Sep 10 17:58:40.747642 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747637 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dadf206-90e5-40f8-aaf1-819dede87a1b" containerName="istio-proxy" Sep 10 17:58:40.747769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747643 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dadf206-90e5-40f8-aaf1-819dede87a1b" containerName="istio-proxy" Sep 10 17:58:40.747769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747704 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" Sep 10 17:58:40.747769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.747713 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dadf206-90e5-40f8-aaf1-819dede87a1b" containerName="istio-proxy" Sep 10 17:58:40.749744 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.749727 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:40.752342 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.752317 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 17:58:40.752466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.752384 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 17:58:40.759445 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.759419 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7"] Sep 10 17:58:40.840262 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.840208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0595da7c-91a9-48f4-95d1-fefafd94674e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7\" (UID: \"0595da7c-91a9-48f4-95d1-fefafd94674e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:40.940770 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.940743 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0595da7c-91a9-48f4-95d1-fefafd94674e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7\" (UID: \"0595da7c-91a9-48f4-95d1-fefafd94674e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:40.941333 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.941306 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:40.941473 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:40.941456 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0595da7c-91a9-48f4-95d1-fefafd94674e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7\" (UID: \"0595da7c-91a9-48f4-95d1-fefafd94674e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:41.062184 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:41.062151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:41.332951 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:41.332919 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7"] Sep 10 17:58:41.337941 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:58:41.337593 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0595da7c_91a9_48f4_95d1_fefafd94674e.slice/crio-6643e873c0ecc1108746dbd9270914c7c4dc723b3a9ce3deb381eb3a81498215 WatchSource:0}: Error finding container 6643e873c0ecc1108746dbd9270914c7c4dc723b3a9ce3deb381eb3a81498215: Status 404 returned error can't find the container with id 6643e873c0ecc1108746dbd9270914c7c4dc723b3a9ce3deb381eb3a81498215 Sep 10 17:58:41.957383 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:41.957349 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" event={"ID":"0595da7c-91a9-48f4-95d1-fefafd94674e","Type":"ContainerStarted","Data":"a6e604ce66e432a8cdb8a6dc0467b8626f9c8a3f5ba11ff7570017157b1664ca"} Sep 10 17:58:41.957746 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:41.957422 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" event={"ID":"0595da7c-91a9-48f4-95d1-fefafd94674e","Type":"ContainerStarted","Data":"6643e873c0ecc1108746dbd9270914c7c4dc723b3a9ce3deb381eb3a81498215"} Sep 10 17:58:41.973994 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:41.973950 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" podStartSLOduration=1.9739375080000001 podStartE2EDuration="1.973937508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:41.972524184 +0000 UTC m=+1050.854249327" watchObservedRunningTime="2026-09-10 17:58:41.973937508 +0000 UTC m=+1050.855662671" Sep 10 17:58:42.963808 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:42.963771 2565 generic.go:356] "Generic (PLEG): container finished" podID="0595da7c-91a9-48f4-95d1-fefafd94674e" containerID="a6e604ce66e432a8cdb8a6dc0467b8626f9c8a3f5ba11ff7570017157b1664ca" exitCode=0 Sep 10 17:58:42.964263 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:42.963853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" event={"ID":"0595da7c-91a9-48f4-95d1-fefafd94674e","Type":"ContainerDied","Data":"a6e604ce66e432a8cdb8a6dc0467b8626f9c8a3f5ba11ff7570017157b1664ca"} Sep 10 17:58:43.028463 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.028435 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:43.089706 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.089677 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:43.264701 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.264632 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:43.554439 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.554359 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw"] Sep 10 17:58:43.554861 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.554846 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9f877a-9860-4d68-b422-565c350d3697" containerName="istio-proxy" Sep 10 17:58:43.556840 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.556822 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.559431 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.559414 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1fb38e82-openshift-default-dockercfg-67hbb\"" Sep 10 17:58:43.571197 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.571175 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw"] Sep 10 17:58:43.578048 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.578027 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:43.667563 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-istio-token\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-envoy\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-certs\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-credential-socket\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667695 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpd8h\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-kube-api-access-jpd8h\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667737 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667724 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676961-d8d7-4205-b022-c239d2925f6d-istiod-ca-cert\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667769 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676961-d8d7-4205-b022-c239d2925f6d-istio-podinfo\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667824 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-socket\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.667927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.667853 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-data\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.768865 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.768836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-socket\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769000 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.768877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-data\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769000 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.768922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-istio-token\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769000 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.768956 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-envoy\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769000 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.768973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-certs\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769101 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-credential-socket\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769176 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769152 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpd8h\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-kube-api-access-jpd8h\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769265 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769203 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676961-d8d7-4205-b022-c239d2925f6d-istiod-ca-cert\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769327 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676961-d8d7-4205-b022-c239d2925f6d-istio-podinfo\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769327 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769305 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-socket\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769429 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769344 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-data\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769533 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769513 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-credential-socket\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769732 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769708 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-certs\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.769850 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.769782 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676961-d8d7-4205-b022-c239d2925f6d-istiod-ca-cert\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.771285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.771258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-envoy\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.771561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.771542 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676961-d8d7-4205-b022-c239d2925f6d-istio-podinfo\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.776632 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.776611 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-istio-token\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.777003 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.776981 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpd8h\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-kube-api-access-jpd8h\") pod \"e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:43.866926 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:43.866870 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:44.093478 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.093456 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:44.138967 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.138907 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw"] Sep 10 17:58:44.142085 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:58:44.142057 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60676961_d8d7_4205_b022_c239d2925f6d.slice/crio-427d9ad4b92afc0a20946446cf85fb3b9282215081c420cd952a9bff5c851283 WatchSource:0}: Error finding container 427d9ad4b92afc0a20946446cf85fb3b9282215081c420cd952a9bff5c851283: Status 404 returned error can't find the container with id 427d9ad4b92afc0a20946446cf85fb3b9282215081c420cd952a9bff5c851283 Sep 10 17:58:44.144352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.144321 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:58:44.144431 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.144382 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:58:44.144471 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.144427 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:58:44.173175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.173154 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0595da7c-91a9-48f4-95d1-fefafd94674e-maas-api-service-ca\") pod \"0595da7c-91a9-48f4-95d1-fefafd94674e\" (UID: \"0595da7c-91a9-48f4-95d1-fefafd94674e\") " Sep 10 17:58:44.173536 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.173516 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0595da7c-91a9-48f4-95d1-fefafd94674e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0595da7c-91a9-48f4-95d1-fefafd94674e" (UID: "0595da7c-91a9-48f4-95d1-fefafd94674e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:58:44.274729 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.274691 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0595da7c-91a9-48f4-95d1-fefafd94674e-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:58:44.313788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.313754 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:44.975606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.975571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" event={"ID":"60676961-d8d7-4205-b022-c239d2925f6d","Type":"ContainerStarted","Data":"c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594"} Sep 10 17:58:44.975815 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.975625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" event={"ID":"60676961-d8d7-4205-b022-c239d2925f6d","Type":"ContainerStarted","Data":"427d9ad4b92afc0a20946446cf85fb3b9282215081c420cd952a9bff5c851283"} Sep 10 17:58:44.976873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.976849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" event={"ID":"0595da7c-91a9-48f4-95d1-fefafd94674e","Type":"ContainerDied","Data":"6643e873c0ecc1108746dbd9270914c7c4dc723b3a9ce3deb381eb3a81498215"} Sep 10 17:58:44.976977 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.976887 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6643e873c0ecc1108746dbd9270914c7c4dc723b3a9ce3deb381eb3a81498215" Sep 10 17:58:44.976977 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.976857 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7" Sep 10 17:58:44.998174 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:44.998133 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podStartSLOduration=1.998121352 podStartE2EDuration="1.998121352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:44.995322659 +0000 UTC m=+1053.877047801" watchObservedRunningTime="2026-09-10 17:58:44.998121352 +0000 UTC m=+1053.879846489" Sep 10 17:58:45.867484 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:45.867215 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:45.873345 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:45.873320 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:45.981881 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:45.981851 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:45.982611 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:45.982592 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:58:46.946986 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:46.946957 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:47.049384 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:47.049352 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:48.236684 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.236650 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:48.338900 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.338865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:48.680310 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.680285 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:48.777285 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.777252 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:48.806377 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.806348 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7"] Sep 10 17:58:48.806914 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.806891 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0595da7c-91a9-48f4-95d1-fefafd94674e" containerName="revoke-keys" Sep 10 17:58:48.806914 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.806911 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0595da7c-91a9-48f4-95d1-fefafd94674e" containerName="revoke-keys" Sep 10 17:58:48.807107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.806980 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0595da7c-91a9-48f4-95d1-fefafd94674e" containerName="revoke-keys" Sep 10 17:58:48.808887 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.808867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:48.811352 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.811333 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1fb38e82\"" Sep 10 17:58:48.811442 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.811355 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1fb38e82-dockercfg-z879k\"" Sep 10 17:58:48.820267 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.820212 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7"] Sep 10 17:58:48.918156 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.918123 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjjf6\" (UniqueName: \"kubernetes.io/projected/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-kube-api-access-qjjf6\") pod \"payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:48.918332 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:48.918293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-plugins-config-volume\") pod \"payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:49.019742 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:49.019672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjjf6\" (UniqueName: \"kubernetes.io/projected/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-kube-api-access-qjjf6\") pod \"payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:49.019860 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:49.019760 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-plugins-config-volume\") pod \"payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:49.020530 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:49.020511 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-plugins-config-volume\") pod \"payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:49.027503 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:49.027484 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjjf6\" (UniqueName: \"kubernetes.io/projected/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-kube-api-access-qjjf6\") pod \"payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:49.120817 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:49.120788 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:49.397991 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:49.397965 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7"] Sep 10 17:58:49.399791 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:58:49.399767 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb060e2f5_cfd7_40e8_97d7_811edd9ee1fc.slice/crio-b1eb887af1446108a050896501d9317a83a7a4ba45b6e889cfbc78afa783bb1e WatchSource:0}: Error finding container b1eb887af1446108a050896501d9317a83a7a4ba45b6e889cfbc78afa783bb1e: Status 404 returned error can't find the container with id b1eb887af1446108a050896501d9317a83a7a4ba45b6e889cfbc78afa783bb1e Sep 10 17:58:50.007048 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:50.007011 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" event={"ID":"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc","Type":"ContainerStarted","Data":"e9aaa107a83d3e4a4270cb71e0173d3e05f042303bc055ae7dfbbd7674c03861"} Sep 10 17:58:50.007208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:50.007064 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" event={"ID":"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc","Type":"ContainerStarted","Data":"b1eb887af1446108a050896501d9317a83a7a4ba45b6e889cfbc78afa783bb1e"} Sep 10 17:58:50.007208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:50.007112 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:58:50.024758 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:50.024718 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" podStartSLOduration=2.024706077 podStartE2EDuration="2.024706077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:50.023599396 +0000 UTC m=+1058.905324537" watchObservedRunningTime="2026-09-10 17:58:50.024706077 +0000 UTC m=+1058.906431229" Sep 10 17:58:50.918650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:50.918610 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:51.465460 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:51.465430 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:51.540524 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:51.540489 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:51.601550 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:51.601519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:51.673665 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:51.673634 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7"] Sep 10 17:58:51.674656 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:51.674633 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:51.675836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:51.675816 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c34c13-gk7v7"] Sep 10 17:58:52.025386 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:52.025354 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:53.622876 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:53.622843 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0595da7c-91a9-48f4-95d1-fefafd94674e" path="/var/lib/kubelet/pods/0595da7c-91a9-48f4-95d1-fefafd94674e/volumes" Sep 10 17:58:58.225583 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:58.225534 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:58.326162 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:58.326132 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:58.481307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:58.478874 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl"] Sep 10 17:58:58.481307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:58.479193 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" containerID="cri-o://2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449" gracePeriod=30 Sep 10 17:58:58.488903 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:58.488867 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:58:58.948081 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:58:58.948040 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:01.015065 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:01.015029 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:59:03.754167 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.754098 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:59:03.863698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863621 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-data\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.863698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863669 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-socket\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.863904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863757 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istiod-ca-cert\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.863904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863836 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-token\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.863904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863872 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-podinfo\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.864075 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863911 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-envoy\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.864075 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863946 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-data" (OuterVolumeSpecName: "istio-data") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:03.864075 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863966 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:03.864075 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.863978 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krr9j\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-kube-api-access-krr9j\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.864317 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864087 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-certs\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.864317 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864117 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-credential-socket\") pod \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\" (UID: \"4cc9f3d1-90a1-4a6a-9350-40f3611b6326\") " Sep 10 17:59:03.864466 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864379 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:03.864521 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864285 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:59:03.864603 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864583 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:03.864650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864638 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.864698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864656 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.864698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864670 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.864698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.864682 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.866718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.866665 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-token" (OuterVolumeSpecName: "istio-token") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:03.866718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.866691 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:59:03.866718 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.866707 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-kube-api-access-krr9j" (OuterVolumeSpecName: "kube-api-access-krr9j") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "kube-api-access-krr9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:03.866900 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.866755 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4cc9f3d1-90a1-4a6a-9350-40f3611b6326" (UID: "4cc9f3d1-90a1-4a6a-9350-40f3611b6326"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:03.965568 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.965537 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.965568 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.965566 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.973472 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.965576 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.973472 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.965585 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krr9j\" (UniqueName: \"kubernetes.io/projected/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-kube-api-access-krr9j\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:03.973472 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:03.965595 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cc9f3d1-90a1-4a6a-9350-40f3611b6326-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:04.109602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.109569 2565 generic.go:356] "Generic (PLEG): container finished" podID="60676961-d8d7-4205-b022-c239d2925f6d" containerID="c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594" exitCode=0 Sep 10 17:59:04.109769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.109647 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" event={"ID":"60676961-d8d7-4205-b022-c239d2925f6d","Type":"ContainerDied","Data":"c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594"} Sep 10 17:59:04.110149 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.110130 2565 scope.go:117] "RemoveContainer" containerID="c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594" Sep 10 17:59:04.111361 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111325 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:04.111480 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111404 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:04.111480 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111442 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:04.111582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111500 2565 generic.go:356] "Generic (PLEG): container finished" podID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerID="2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449" exitCode=0 Sep 10 17:59:04.111582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111529 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" event={"ID":"4cc9f3d1-90a1-4a6a-9350-40f3611b6326","Type":"ContainerDied","Data":"2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449"} Sep 10 17:59:04.111582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111556 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" Sep 10 17:59:04.111582 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111569 2565 scope.go:117] "RemoveContainer" containerID="2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449" Sep 10 17:59:04.111752 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.111560 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl" event={"ID":"4cc9f3d1-90a1-4a6a-9350-40f3611b6326","Type":"ContainerDied","Data":"894b8212b73a8effb8c5dc3986a46cca8e0ba082770f01bb3c313cf8aba30893"} Sep 10 17:59:04.123551 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.123534 2565 scope.go:117] "RemoveContainer" containerID="a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509" Sep 10 17:59:04.136904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.136513 2565 scope.go:117] "RemoveContainer" containerID="2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449" Sep 10 17:59:04.137016 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:59:04.136977 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449\": container with ID starting with 2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449 not found: ID does not exist" containerID="2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449" Sep 10 17:59:04.137089 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.137013 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449"} err="failed to get container status \"2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449\": rpc error: code = NotFound desc = could not find container \"2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449\": container with ID starting with 2610544a8ddc7a4ab286191943dd9fd18f5d7a5e422ca21a838115499eba6449 not found: ID does not exist" Sep 10 17:59:04.137089 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.137039 2565 scope.go:117] "RemoveContainer" containerID="a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509" Sep 10 17:59:04.137668 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:59:04.137606 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509\": container with ID starting with a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509 not found: ID does not exist" containerID="a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509" Sep 10 17:59:04.137668 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.137640 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509"} err="failed to get container status \"a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509\": rpc error: code = NotFound desc = could not find container \"a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509\": container with ID starting with a89a04985d12706889cfabb082cca4bcafdbe1de1b2088c9fc0776df90cd5509 not found: ID does not exist" Sep 10 17:59:04.147117 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.146523 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl"] Sep 10 17:59:04.153701 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:04.153672 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c34c13-openshift-default-578c8cc48-66frl"] Sep 10 17:59:05.119029 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:05.118991 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" event={"ID":"60676961-d8d7-4205-b022-c239d2925f6d","Type":"ContainerStarted","Data":"a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7"} Sep 10 17:59:05.623324 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:05.623281 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" path="/var/lib/kubelet/pods/4cc9f3d1-90a1-4a6a-9350-40f3611b6326/volumes" Sep 10 17:59:05.867530 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:05.867488 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:59:05.867788 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:05.867757 2565 patch_prober.go:28] interesting pod/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Sep 10 17:59:05.867915 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:05.867822 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Sep 10 17:59:06.867467 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:06.867425 2565 patch_prober.go:28] interesting pod/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Sep 10 17:59:06.867979 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:06.867499 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Sep 10 17:59:07.867563 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:07.867523 2565 patch_prober.go:28] interesting pod/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Sep 10 17:59:07.868138 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:07.868106 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Sep 10 17:59:08.867576 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:08.867542 2565 patch_prober.go:28] interesting pod/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Sep 10 17:59:08.868162 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:08.868123 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Sep 10 17:59:09.416849 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:09.416818 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:09.867555 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:09.867516 2565 patch_prober.go:28] interesting pod/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Sep 10 17:59:09.867853 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:09.867823 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Sep 10 17:59:10.109735 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.109687 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:10.206322 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.206288 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:10.301218 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.301177 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:10.361449 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.361405 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:10.747461 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.747423 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:10.847656 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.847618 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:10.870693 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.870670 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:59:10.871087 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.871042 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:59:10.871628 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:10.871610 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:59:11.870942 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:11.870908 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:12.525669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:12.525634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:12.624578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:12.624541 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:12.956694 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:12.956658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:16.627338 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:16.627297 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:16.680993 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:16.680960 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:16.851798 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:16.851763 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp"] Sep 10 17:59:16.852039 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:16.852013 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" containerID="cri-o://c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a" gracePeriod=30 Sep 10 17:59:16.876297 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:16.876220 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:17.439335 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:17.439300 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:17.744105 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:17.744030 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:18.040032 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:18.039933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:18.912151 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:18.912115 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:19.012512 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:19.012482 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:22.101395 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.101371 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:59:22.219904 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.219873 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5573d4b-300d-4179-81f5-108f34307c9d" containerID="c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a" exitCode=0 Sep 10 17:59:22.220090 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.219921 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" Sep 10 17:59:22.220090 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.219971 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" event={"ID":"d5573d4b-300d-4179-81f5-108f34307c9d","Type":"ContainerDied","Data":"c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a"} Sep 10 17:59:22.220090 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.220030 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp" event={"ID":"d5573d4b-300d-4179-81f5-108f34307c9d","Type":"ContainerDied","Data":"2894eee1a670f748817d4542f16a022b1fee7c0850844efb0a88f192c3a8a66c"} Sep 10 17:59:22.220090 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.220060 2565 scope.go:117] "RemoveContainer" containerID="c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a" Sep 10 17:59:22.228680 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.228664 2565 scope.go:117] "RemoveContainer" containerID="2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a" Sep 10 17:59:22.238404 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238381 2565 scope.go:117] "RemoveContainer" containerID="c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a" Sep 10 17:59:22.238514 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238455 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5573d4b-300d-4179-81f5-108f34307c9d-istiod-ca-cert\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238522 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-envoy\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238560 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-certs\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238623 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-socket\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238652 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-credential-socket\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238806 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238715 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-data\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238806 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238767 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-istio-token\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238806 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238794 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5573d4b-300d-4179-81f5-108f34307c9d-istio-podinfo\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238846 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8d7z\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-kube-api-access-f8d7z\") pod \"d5573d4b-300d-4179-81f5-108f34307c9d\" (UID: \"d5573d4b-300d-4179-81f5-108f34307c9d\") " Sep 10 17:59:22.238956 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:59:22.238884 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a\": container with ID starting with c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a not found: ID does not exist" containerID="c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a" Sep 10 17:59:22.238956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238922 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a"} err="failed to get container status \"c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a\": rpc error: code = NotFound desc = could not find container \"c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a\": container with ID starting with c3527d6f6a75d4a7759e0af117fd0fda41f7eeee0f27a46b54cefcfbc6b93c0a not found: ID does not exist" Sep 10 17:59:22.238956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238947 2565 scope.go:117] "RemoveContainer" containerID="2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a" Sep 10 17:59:22.238956 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238947 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:22.239201 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.238968 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5573d4b-300d-4179-81f5-108f34307c9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:59:22.239400 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239198 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:22.239400 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239246 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:22.239575 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239550 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:22.239686 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:59:22.239656 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a\": container with ID starting with 2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a not found: ID does not exist" containerID="2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a" Sep 10 17:59:22.239856 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239826 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a"} err="failed to get container status \"2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a\": rpc error: code = NotFound desc = could not find container \"2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a\": container with ID starting with 2a8c7ef09a6863d6c3bbd297abd0d117b8be3936c3332ed985c941c9eb2da24a not found: ID does not exist" Sep 10 17:59:22.239856 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239706 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5573d4b-300d-4179-81f5-108f34307c9d-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.240040 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239873 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.240040 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239889 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.240040 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239903 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.240040 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.239918 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.241084 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.241062 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5573d4b-300d-4179-81f5-108f34307c9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:59:22.241217 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.241201 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:22.241697 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.241675 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-kube-api-access-f8d7z" (OuterVolumeSpecName: "kube-api-access-f8d7z") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "kube-api-access-f8d7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:22.241771 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.241705 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5573d4b-300d-4179-81f5-108f34307c9d" (UID: "d5573d4b-300d-4179-81f5-108f34307c9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:22.301547 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.301524 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:22.340873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.340848 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.340873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.340871 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5573d4b-300d-4179-81f5-108f34307c9d-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.340996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.340885 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8d7z\" (UniqueName: \"kubernetes.io/projected/d5573d4b-300d-4179-81f5-108f34307c9d-kube-api-access-f8d7z\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.340996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.340898 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5573d4b-300d-4179-81f5-108f34307c9d-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:22.351175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.351150 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:22.543275 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.543242 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp"] Sep 10 17:59:22.548619 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:22.548596 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-18490e-openshift-default-7f586c8cc8-gx9cp"] Sep 10 17:59:23.625043 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:23.624998 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" path="/var/lib/kubelet/pods/d5573d4b-300d-4179-81f5-108f34307c9d/volumes" Sep 10 17:59:23.721095 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:23.721058 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:23.812839 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:23.812805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:25.022721 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:25.022685 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:25.122646 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:25.122616 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:25.178310 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:25.178277 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:25.479093 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:25.479062 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:25.579665 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:25.579630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:29.445478 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:29.445442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:30.007314 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.007282 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:30.031208 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.031174 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7"] Sep 10 17:59:30.031435 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.031409 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" podUID="b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" containerName="payload-processing" containerID="cri-o://e9aaa107a83d3e4a4270cb71e0173d3e05f042303bc055ae7dfbbd7674c03861" gracePeriod=30 Sep 10 17:59:30.067859 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.067833 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:30.121602 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.121577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:30.220698 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.220665 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:30.261834 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.261804 2565 generic.go:356] "Generic (PLEG): container finished" podID="b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" containerID="e9aaa107a83d3e4a4270cb71e0173d3e05f042303bc055ae7dfbbd7674c03861" exitCode=0 Sep 10 17:59:30.261945 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.261853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" event={"ID":"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc","Type":"ContainerDied","Data":"e9aaa107a83d3e4a4270cb71e0173d3e05f042303bc055ae7dfbbd7674c03861"} Sep 10 17:59:30.281477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.281448 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:59:30.310011 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.309980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:30.410515 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.410482 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjjf6\" (UniqueName: \"kubernetes.io/projected/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-kube-api-access-qjjf6\") pod \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " Sep 10 17:59:30.410706 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.410571 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-plugins-config-volume\") pod \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\" (UID: \"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc\") " Sep 10 17:59:30.410988 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.410961 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" (UID: "b060e2f5-cfd7-40e8-97d7-811edd9ee1fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:59:30.412590 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.412557 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-kube-api-access-qjjf6" (OuterVolumeSpecName: "kube-api-access-qjjf6") pod "b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" (UID: "b060e2f5-cfd7-40e8-97d7-811edd9ee1fc"). InnerVolumeSpecName "kube-api-access-qjjf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:30.511888 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.511814 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjjf6\" (UniqueName: \"kubernetes.io/projected/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-kube-api-access-qjjf6\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:30.511888 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.511841 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:30.577346 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:30.577314 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:31.268927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:31.268891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" event={"ID":"b060e2f5-cfd7-40e8-97d7-811edd9ee1fc","Type":"ContainerDied","Data":"b1eb887af1446108a050896501d9317a83a7a4ba45b6e889cfbc78afa783bb1e"} Sep 10 17:59:31.268927 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:31.268913 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7" Sep 10 17:59:31.269165 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:31.268952 2565 scope.go:117] "RemoveContainer" containerID="e9aaa107a83d3e4a4270cb71e0173d3e05f042303bc055ae7dfbbd7674c03861" Sep 10 17:59:31.291596 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:31.291570 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7"] Sep 10 17:59:31.296919 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:31.296896 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1fb38e82-5ddb957457-2rhm7"] Sep 10 17:59:31.623081 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:31.622995 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" path="/var/lib/kubelet/pods/b060e2f5-cfd7-40e8-97d7-811edd9ee1fc/volumes" Sep 10 17:59:36.500640 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:36.500604 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:36.561896 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:36.561856 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:36.747833 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:36.747798 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw"] Sep 10 17:59:36.748107 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:36.748080 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" containerID="cri-o://a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7" gracePeriod=30 Sep 10 17:59:36.770296 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:36.770185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:37.120650 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:37.120569 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:37.222830 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:37.222781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:37.330769 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:37.330728 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:40.874275 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:40.874216 2565 patch_prober.go:28] interesting pod/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 17:59:40.874664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:40.874294 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:59:42.002453 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.002426 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:59:42.123360 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123269 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676961-d8d7-4205-b022-c239d2925f6d-istio-podinfo\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123360 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123321 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-envoy\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123360 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123345 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-socket\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123378 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-credential-socket\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123436 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676961-d8d7-4205-b022-c239d2925f6d-istiod-ca-cert\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123477 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-certs\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123511 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpd8h\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-kube-api-access-jpd8h\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123545 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-data\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123664 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123599 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-istio-token\") pod \"60676961-d8d7-4205-b022-c239d2925f6d\" (UID: \"60676961-d8d7-4205-b022-c239d2925f6d\") " Sep 10 17:59:42.123975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123671 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:42.123975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123705 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:42.123975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123771 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:42.123975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123879 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60676961-d8d7-4205-b022-c239d2925f6d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:59:42.123975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123895 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-data" (OuterVolumeSpecName: "istio-data") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:42.124203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.123991 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.124203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.124011 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.124203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.124026 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.124203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.124041 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60676961-d8d7-4205-b022-c239d2925f6d-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.124203 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.124054 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.125643 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.125616 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60676961-d8d7-4205-b022-c239d2925f6d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 17:59:42.125765 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.125694 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-kube-api-access-jpd8h" (OuterVolumeSpecName: "kube-api-access-jpd8h") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "kube-api-access-jpd8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:42.125765 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.125727 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-istio-token" (OuterVolumeSpecName: "istio-token") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:59:42.125854 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.125758 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60676961-d8d7-4205-b022-c239d2925f6d" (UID: "60676961-d8d7-4205-b022-c239d2925f6d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:59:42.224836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.224794 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpd8h\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-kube-api-access-jpd8h\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.224836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.224825 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60676961-d8d7-4205-b022-c239d2925f6d-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.224836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.224836 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60676961-d8d7-4205-b022-c239d2925f6d-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.224836 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.224844 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60676961-d8d7-4205-b022-c239d2925f6d-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 17:59:42.326762 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.326724 2565 generic.go:356] "Generic (PLEG): container finished" podID="60676961-d8d7-4205-b022-c239d2925f6d" containerID="a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7" exitCode=0 Sep 10 17:59:42.326926 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.326768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" event={"ID":"60676961-d8d7-4205-b022-c239d2925f6d","Type":"ContainerDied","Data":"a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7"} Sep 10 17:59:42.326926 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.326779 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" Sep 10 17:59:42.326926 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.326866 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw" event={"ID":"60676961-d8d7-4205-b022-c239d2925f6d","Type":"ContainerDied","Data":"427d9ad4b92afc0a20946446cf85fb3b9282215081c420cd952a9bff5c851283"} Sep 10 17:59:42.326926 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.326897 2565 scope.go:117] "RemoveContainer" containerID="a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7" Sep 10 17:59:42.337380 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.337363 2565 scope.go:117] "RemoveContainer" containerID="c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594" Sep 10 17:59:42.345566 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.345545 2565 scope.go:117] "RemoveContainer" containerID="a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7" Sep 10 17:59:42.345850 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:59:42.345834 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7\": container with ID starting with a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7 not found: ID does not exist" containerID="a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7" Sep 10 17:59:42.345890 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.345861 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7"} err="failed to get container status \"a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7\": rpc error: code = NotFound desc = could not find container \"a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7\": container with ID starting with a4e77fb8332c3ee09904232b9c583aed94341b1509246635c19803e128de5dd7 not found: ID does not exist" Sep 10 17:59:42.345890 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.345878 2565 scope.go:117] "RemoveContainer" containerID="c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594" Sep 10 17:59:42.346115 ip-10-0-134-57 kubenswrapper[2565]: E0910 17:59:42.346099 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594\": container with ID starting with c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594 not found: ID does not exist" containerID="c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594" Sep 10 17:59:42.346151 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.346120 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594"} err="failed to get container status \"c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594\": rpc error: code = NotFound desc = could not find container \"c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594\": container with ID starting with c11ea11e9e35f26e2ac04eb59fdd76183ea1fad164c415dfb4e23575da804594 not found: ID does not exist" Sep 10 17:59:42.349467 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.349443 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw"] Sep 10 17:59:42.353300 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:42.353277 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1fb38e82-openshift-default-5b8768fd84-6ffdw"] Sep 10 17:59:43.219334 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:43.219302 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:43.276676 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:43.276648 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:43.368175 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:43.368136 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:43.465723 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:43.465685 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:43.563646 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:43.563568 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:43.622684 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:43.622645 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60676961-d8d7-4205-b022-c239d2925f6d" path="/var/lib/kubelet/pods/60676961-d8d7-4205-b022-c239d2925f6d/volumes" Sep 10 17:59:45.905891 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:45.905854 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:48.670024 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:48.669989 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:49.008746 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.008658 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:49.269817 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.269736 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5"] Sep 10 17:59:49.270209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270183 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" Sep 10 17:59:49.270209 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270207 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270220 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" containerName="payload-processing" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270246 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" containerName="payload-processing" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270262 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270270 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270279 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270284 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270295 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270301 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270315 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" Sep 10 17:59:49.270393 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270320 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" Sep 10 17:59:49.270792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270410 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" Sep 10 17:59:49.270792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270419 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" Sep 10 17:59:49.270792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270426 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="60676961-d8d7-4205-b022-c239d2925f6d" containerName="istio-proxy" Sep 10 17:59:49.270792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270433 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b060e2f5-cfd7-40e8-97d7-811edd9ee1fc" containerName="payload-processing" Sep 10 17:59:49.270792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270438 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" Sep 10 17:59:49.270792 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.270444 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" Sep 10 17:59:49.273683 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.273666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.276728 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.276701 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-fc23cf-openshift-default-dockercfg-k675v\"" Sep 10 17:59:49.287331 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.287305 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5"] Sep 10 17:59:49.287448 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.287427 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:49.379895 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.379867 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:49.385888 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.385866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-envoy\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.385996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.385906 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-data\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.385996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.385970 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-socket\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.385996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.385987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-credential-socket\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.386099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.386048 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-podinfo\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.386099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.386076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-token\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.386099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.386095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4mmc\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-kube-api-access-c4mmc\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.386245 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.386136 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-certs\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.386245 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.386203 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb047b72-2959-4dcc-953d-41e41b4d298b-istiod-ca-cert\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.486928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.486897 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-podinfo\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.486928 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.486931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-token\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.486950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4mmc\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-kube-api-access-c4mmc\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.486968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-certs\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb047b72-2959-4dcc-953d-41e41b4d298b-istiod-ca-cert\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487187 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-envoy\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487407 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487205 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-data\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487407 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487358 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-socket\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487407 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487388 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-credential-socket\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487567 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487544 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-certs\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487662 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487641 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-data\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487723 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487702 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-socket\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487768 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487753 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-credential-socket\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.487882 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.487865 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb047b72-2959-4dcc-953d-41e41b4d298b-istiod-ca-cert\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.489429 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.489402 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-envoy\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.489606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.489587 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-podinfo\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.494871 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.494848 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-token\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.495115 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.495097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4mmc\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-kube-api-access-c4mmc\") pod \"e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.583733 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.583679 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:49.864133 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.864106 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5"] Sep 10 17:59:49.865616 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:59:49.865584 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb047b72_2959_4dcc_953d_41e41b4d298b.slice/crio-ad707090e46929c5cf78eab7f4bec1fdf972ca1ecd9275c62be4ebf6a3abdab6 WatchSource:0}: Error finding container ad707090e46929c5cf78eab7f4bec1fdf972ca1ecd9275c62be4ebf6a3abdab6: Status 404 returned error can't find the container with id ad707090e46929c5cf78eab7f4bec1fdf972ca1ecd9275c62be4ebf6a3abdab6 Sep 10 17:59:49.867908 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.867865 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:49.867995 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.867974 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:49.868045 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.868027 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:49.922169 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:49.922140 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:50.034483 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.034436 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:50.372091 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.372055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" event={"ID":"eb047b72-2959-4dcc-953d-41e41b4d298b","Type":"ContainerStarted","Data":"36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d"} Sep 10 17:59:50.372273 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.372105 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" event={"ID":"eb047b72-2959-4dcc-953d-41e41b4d298b","Type":"ContainerStarted","Data":"ad707090e46929c5cf78eab7f4bec1fdf972ca1ecd9275c62be4ebf6a3abdab6"} Sep 10 17:59:50.395578 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.395529 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podStartSLOduration=1.395513799 podStartE2EDuration="1.395513799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:50.394828133 +0000 UTC m=+1119.276553295" watchObservedRunningTime="2026-09-10 17:59:50.395513799 +0000 UTC m=+1119.277238941" Sep 10 17:59:50.584477 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.584450 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:50.589404 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.589379 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:50.816462 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:50.816420 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:51.111858 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.111767 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj"] Sep 10 17:59:51.112369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.112191 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" Sep 10 17:59:51.112369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.112203 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5573d4b-300d-4179-81f5-108f34307c9d" containerName="istio-proxy" Sep 10 17:59:51.112369 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.112302 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc9f3d1-90a1-4a6a-9350-40f3611b6326" containerName="istio-proxy" Sep 10 17:59:51.114378 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.114358 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.117119 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.117098 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-971a94-openshift-default-dockercfg-cd4m7\"" Sep 10 17:59:51.127320 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.127283 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj"] Sep 10 17:59:51.131614 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.131594 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:51.206996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.206957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-data\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.206996 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-podinfo\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207250 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istiod-ca-cert\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207250 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-socket\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207250 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-certs\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207433 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-credential-socket\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207433 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-envoy\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207433 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88g82\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-kube-api-access-88g82\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.207594 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.207470 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-token\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.269299 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.269266 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:51.308479 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308444 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-data\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-podinfo\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308509 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istiod-ca-cert\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308532 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-socket\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308606 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308586 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-certs\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308816 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-credential-socket\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308816 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308767 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-envoy\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308816 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88g82\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-kube-api-access-88g82\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308965 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-token\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.308965 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308914 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-data\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.309072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.308962 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-socket\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.309307 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.309286 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-certs\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.309401 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.309330 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-credential-socket\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.309401 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.309367 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istiod-ca-cert\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.310983 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.310960 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-envoy\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.311054 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.311040 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-podinfo\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.320172 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.320149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-token\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.320316 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.320244 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88g82\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-kube-api-access-88g82\") pod \"e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.377783 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.377720 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:51.379028 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.378998 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 17:59:51.426156 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.426129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:51.711829 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.711805 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj"] Sep 10 17:59:51.713883 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:59:51.713859 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a54ca7_2ec3_4bfd_9bf4_a546e34a13af.slice/crio-6dda6211ff82d9ef28d47c37a230655244f0f6963ed58d3373297164f9a09f4c WatchSource:0}: Error finding container 6dda6211ff82d9ef28d47c37a230655244f0f6963ed58d3373297164f9a09f4c: Status 404 returned error can't find the container with id 6dda6211ff82d9ef28d47c37a230655244f0f6963ed58d3373297164f9a09f4c Sep 10 17:59:51.716330 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.716296 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:51.716437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.716374 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:51.716437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:51.716423 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 17:59:52.356647 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.356612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:52.383873 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.383840 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" event={"ID":"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af","Type":"ContainerStarted","Data":"4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606"} Sep 10 17:59:52.384073 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.383903 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" event={"ID":"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af","Type":"ContainerStarted","Data":"6dda6211ff82d9ef28d47c37a230655244f0f6963ed58d3373297164f9a09f4c"} Sep 10 17:59:52.408868 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.408820 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podStartSLOduration=1.4088048020000001 podStartE2EDuration="1.408804802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:52.408048654 +0000 UTC m=+1121.289773821" watchObservedRunningTime="2026-09-10 17:59:52.408804802 +0000 UTC m=+1121.290529943" Sep 10 17:59:52.426500 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.426472 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:52.431569 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.431549 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:52.455414 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:52.455394 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:53.389172 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:53.389136 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:53.390072 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:53.390055 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 17:59:53.538716 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:53.538682 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:53.636692 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:53.636659 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:53.930673 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:53.930639 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:54.099010 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:54.098978 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:54.196099 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:54.196022 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:54.312376 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:54.312347 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:54.397437 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:54.397406 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:55.474854 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:55.474821 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:55.574205 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:55.574177 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:55.717908 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:55.717872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:56.046743 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.046716 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:56.146975 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.146944 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 17:59:56.160130 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.160102 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls"] Sep 10 17:59:56.162669 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.162651 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.166561 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.166543 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-971a94\"" Sep 10 17:59:56.166651 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.166641 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-971a94-dockercfg-vv9rp\"" Sep 10 17:59:56.188462 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.188430 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls"] Sep 10 17:59:56.258810 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.258776 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5npj4\" (UniqueName: \"kubernetes.io/projected/ccaf994e-3c89-4158-a517-d68235721318-kube-api-access-5npj4\") pod \"payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.259137 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.259109 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccaf994e-3c89-4158-a517-d68235721318-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.359971 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.359886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccaf994e-3c89-4158-a517-d68235721318-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.360110 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.360041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5npj4\" (UniqueName: \"kubernetes.io/projected/ccaf994e-3c89-4158-a517-d68235721318-kube-api-access-5npj4\") pod \"payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.360710 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.360688 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccaf994e-3c89-4158-a517-d68235721318-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.373432 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.373410 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5npj4\" (UniqueName: \"kubernetes.io/projected/ccaf994e-3c89-4158-a517-d68235721318-kube-api-access-5npj4\") pod \"payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.473214 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.473188 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:56.742541 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:56.742513 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls"] Sep 10 17:59:56.744740 ip-10-0-134-57 kubenswrapper[2565]: W0910 17:59:56.744711 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccaf994e_3c89_4158_a517_d68235721318.slice/crio-43a59ceb5a020dadd283867ffa0076431e8c216ab392f3310fc55ba5e74ec857 WatchSource:0}: Error finding container 43a59ceb5a020dadd283867ffa0076431e8c216ab392f3310fc55ba5e74ec857: Status 404 returned error can't find the container with id 43a59ceb5a020dadd283867ffa0076431e8c216ab392f3310fc55ba5e74ec857 Sep 10 17:59:57.412122 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:57.412085 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" event={"ID":"ccaf994e-3c89-4158-a517-d68235721318","Type":"ContainerStarted","Data":"00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc"} Sep 10 17:59:57.412359 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:57.412143 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" event={"ID":"ccaf994e-3c89-4158-a517-d68235721318","Type":"ContainerStarted","Data":"43a59ceb5a020dadd283867ffa0076431e8c216ab392f3310fc55ba5e74ec857"} Sep 10 17:59:57.412359 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:57.412188 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 17:59:57.430620 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:57.430573 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" podStartSLOduration=1.4305596170000001 podStartE2EDuration="1.430559617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:57.428543572 +0000 UTC m=+1126.310268714" watchObservedRunningTime="2026-09-10 17:59:57.430559617 +0000 UTC m=+1126.312284822" Sep 10 17:59:59.542155 ip-10-0-134-57 kubenswrapper[2565]: I0910 17:59:59.542122 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:00.155655 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.155620 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc"] Sep 10 18:00:00.159235 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.159196 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" Sep 10 18:00:00.159454 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.159431 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr"] Sep 10 18:00:00.161764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.161748 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" Sep 10 18:00:00.164399 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.164375 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:00:00.169276 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.169254 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc"] Sep 10 18:00:00.175751 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.175728 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr"] Sep 10 18:00:00.178733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.178710 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" Sep 10 18:00:00.185341 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.185318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" Sep 10 18:00:00.475245 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.474475 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc"] Sep 10 18:00:00.481288 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:00:00.480834 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e36c5fc_5b5c_44e4_91ec_a04051036159.slice/crio-2c2b2ac52c99f5a73513f75ea37dbe3f8147717d935164b0123d844771784618 WatchSource:0}: Error finding container 2c2b2ac52c99f5a73513f75ea37dbe3f8147717d935164b0123d844771784618: Status 404 returned error can't find the container with id 2c2b2ac52c99f5a73513f75ea37dbe3f8147717d935164b0123d844771784618 Sep 10 18:00:00.499867 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:00.499841 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr"] Sep 10 18:00:00.501253 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:00:00.501215 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6481e1d_d625_4945_a5bf_9f3aaae7fff2.slice/crio-ee760f4426c86d0f24189fc0a9b8d119426f96625ab3de498adb4c70eec7f23c WatchSource:0}: Error finding container ee760f4426c86d0f24189fc0a9b8d119426f96625ab3de498adb4c70eec7f23c: Status 404 returned error can't find the container with id ee760f4426c86d0f24189fc0a9b8d119426f96625ab3de498adb4c70eec7f23c Sep 10 18:00:01.437212 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:01.437167 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" event={"ID":"4e36c5fc-5b5c-44e4-91ec-a04051036159","Type":"ContainerStarted","Data":"867513d0e56282f5907786f7fc6e022f4178a4106ce040ce6e88a10d28466709"} Sep 10 18:00:01.437684 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:01.437251 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" event={"ID":"4e36c5fc-5b5c-44e4-91ec-a04051036159","Type":"ContainerStarted","Data":"2c2b2ac52c99f5a73513f75ea37dbe3f8147717d935164b0123d844771784618"} Sep 10 18:00:01.438889 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:01.438845 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" event={"ID":"b6481e1d-d625-4945-a5bf-9f3aaae7fff2","Type":"ContainerStarted","Data":"612a3a2b72a7e74fa3d5b96d273b7247305d62e709a117c72b057ba3f18d8ce2"} Sep 10 18:00:01.438972 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:01.438918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" event={"ID":"b6481e1d-d625-4945-a5bf-9f3aaae7fff2","Type":"ContainerStarted","Data":"ee760f4426c86d0f24189fc0a9b8d119426f96625ab3de498adb4c70eec7f23c"} Sep 10 18:00:01.520948 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:01.520895 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" podStartSLOduration=1.520875109 podStartE2EDuration="1.520875109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:01.520600064 +0000 UTC m=+1130.402325209" watchObservedRunningTime="2026-09-10 18:00:01.520875109 +0000 UTC m=+1130.402600249" Sep 10 18:00:01.522517 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:01.522476 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" podStartSLOduration=1.522465146 podStartE2EDuration="1.522465146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:01.495434873 +0000 UTC m=+1130.377160015" watchObservedRunningTime="2026-09-10 18:00:01.522465146 +0000 UTC m=+1130.404190286" Sep 10 18:00:02.445714 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:02.445675 2565 generic.go:356] "Generic (PLEG): container finished" podID="b6481e1d-d625-4945-a5bf-9f3aaae7fff2" containerID="612a3a2b72a7e74fa3d5b96d273b7247305d62e709a117c72b057ba3f18d8ce2" exitCode=0 Sep 10 18:00:02.446086 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:02.445775 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" event={"ID":"b6481e1d-d625-4945-a5bf-9f3aaae7fff2","Type":"ContainerDied","Data":"612a3a2b72a7e74fa3d5b96d273b7247305d62e709a117c72b057ba3f18d8ce2"} Sep 10 18:00:02.447598 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:02.447573 2565 generic.go:356] "Generic (PLEG): container finished" podID="4e36c5fc-5b5c-44e4-91ec-a04051036159" containerID="867513d0e56282f5907786f7fc6e022f4178a4106ce040ce6e88a10d28466709" exitCode=0 Sep 10 18:00:02.447696 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:02.447649 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" event={"ID":"4e36c5fc-5b5c-44e4-91ec-a04051036159","Type":"ContainerDied","Data":"867513d0e56282f5907786f7fc6e022f4178a4106ce040ce6e88a10d28466709"} Sep 10 18:00:03.717605 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:03.717583 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" Sep 10 18:00:03.720492 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:03.720473 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" Sep 10 18:00:04.466361 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:04.466318 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" event={"ID":"b6481e1d-d625-4945-a5bf-9f3aaae7fff2","Type":"ContainerDied","Data":"ee760f4426c86d0f24189fc0a9b8d119426f96625ab3de498adb4c70eec7f23c"} Sep 10 18:00:04.466581 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:04.466386 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee760f4426c86d0f24189fc0a9b8d119426f96625ab3de498adb4c70eec7f23c" Sep 10 18:00:04.466581 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:04.466462 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr" Sep 10 18:00:04.472116 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:04.472077 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" event={"ID":"4e36c5fc-5b5c-44e4-91ec-a04051036159","Type":"ContainerDied","Data":"2c2b2ac52c99f5a73513f75ea37dbe3f8147717d935164b0123d844771784618"} Sep 10 18:00:04.472297 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:04.472139 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c2b2ac52c99f5a73513f75ea37dbe3f8147717d935164b0123d844771784618" Sep 10 18:00:04.472297 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:04.472203 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc" Sep 10 18:00:08.419952 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:08.419908 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 18:00:08.921349 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:08.921317 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:58380->10.133.0.57:15021: read: connection reset by peer" start-of-body= Sep 10 18:00:08.921523 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:08.921403 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:58380->10.133.0.57:15021: read: connection reset by peer" Sep 10 18:00:09.511458 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:09.511425 2565 generic.go:356] "Generic (PLEG): container finished" podID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerID="4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606" exitCode=0 Sep 10 18:00:09.511948 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:09.511474 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" event={"ID":"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af","Type":"ContainerDied","Data":"4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606"} Sep 10 18:00:09.512020 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:09.512001 2565 scope.go:117] "RemoveContainer" containerID="4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606" Sep 10 18:00:09.513245 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:09.513192 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:09.513319 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:09.513282 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:09.513359 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:09.513318 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:10.283351 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.283318 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:10.518238 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.518203 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerID="36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d" exitCode=0 Sep 10 18:00:10.518660 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.518278 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" event={"ID":"eb047b72-2959-4dcc-953d-41e41b4d298b","Type":"ContainerDied","Data":"36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d"} Sep 10 18:00:10.518767 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.518741 2565 scope.go:117] "RemoveContainer" containerID="36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d" Sep 10 18:00:10.519979 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.519934 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:10.520051 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.520023 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:10.520091 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.520057 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:10.520372 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:10.520353 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" event={"ID":"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af","Type":"ContainerStarted","Data":"d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba"} Sep 10 18:00:11.426814 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.426776 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 18:00:11.427005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.426978 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 10 18:00:11.427053 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.427034 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 10 18:00:11.531004 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.530968 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" event={"ID":"eb047b72-2959-4dcc-953d-41e41b4d298b","Type":"ContainerStarted","Data":"95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6"} Sep 10 18:00:11.584906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.584865 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 18:00:11.585280 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.585256 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 10 18:00:11.585501 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:11.585455 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 10 18:00:12.426669 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:12.426627 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 10 18:00:12.426848 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:12.426694 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 10 18:00:12.584540 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:12.584501 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 10 18:00:12.585121 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:12.585087 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 10 18:00:13.427323 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:13.427286 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 10 18:00:13.427514 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:13.427349 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 10 18:00:13.585055 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:13.585004 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 10 18:00:13.585653 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:13.585621 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 10 18:00:14.427158 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:14.427120 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 10 18:00:14.427369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:14.427176 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 10 18:00:14.585080 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:14.585036 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 10 18:00:14.585581 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:14.585105 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 10 18:00:14.629210 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:14.629170 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:15.427458 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:15.427419 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 10 18:00:15.427642 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:15.427481 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 10 18:00:15.584982 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:15.584950 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 10 18:00:15.585165 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:15.585004 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 10 18:00:16.430216 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:16.430186 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 18:00:16.430567 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:16.430541 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 18:00:16.431376 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:16.431360 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 18:00:16.584339 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:16.584305 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 10 18:00:16.584512 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:16.584355 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 10 18:00:16.746658 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:16.746541 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:17.229129 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:17.229096 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:17.587809 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:17.587741 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 18:00:17.588106 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:17.588083 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 18:00:17.588795 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:17.588773 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 18:00:19.269576 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.269539 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:19.592301 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.592185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:19.867860 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.867780 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl"] Sep 10 18:00:19.868413 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.868385 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6481e1d-d625-4945-a5bf-9f3aaae7fff2" containerName="cleanup" Sep 10 18:00:19.868413 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.868410 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6481e1d-d625-4945-a5bf-9f3aaae7fff2" containerName="cleanup" Sep 10 18:00:19.868564 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.868426 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e36c5fc-5b5c-44e4-91ec-a04051036159" containerName="cleanup" Sep 10 18:00:19.868564 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.868434 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e36c5fc-5b5c-44e4-91ec-a04051036159" containerName="cleanup" Sep 10 18:00:19.868655 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.868549 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e36c5fc-5b5c-44e4-91ec-a04051036159" containerName="cleanup" Sep 10 18:00:19.868655 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.868587 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6481e1d-d625-4945-a5bf-9f3aaae7fff2" containerName="cleanup" Sep 10 18:00:19.871504 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.871488 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:19.874370 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.874351 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-09fbb5-openshift-default-dockercfg-gqdb4\"" Sep 10 18:00:19.886106 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.886081 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl"] Sep 10 18:00:19.905464 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:19.905427 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:20.001383 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001352 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-certs\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001516 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-credential-socket\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001564 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001521 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab242f-9d81-409d-abda-7577f121451c-istio-podinfo\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001564 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab242f-9d81-409d-abda-7577f121451c-istiod-ca-cert\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001650 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001576 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-istio-token\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001696 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001649 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-data\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001740 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001702 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjr5\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-kube-api-access-cdjr5\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001740 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001724 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-envoy\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.001823 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.001745 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-socket\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.006020 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.005988 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:20.102908 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.102874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-istio-token\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103093 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.102918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-data\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103178 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103086 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjr5\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-kube-api-access-cdjr5\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103178 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-envoy\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103178 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103146 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-socket\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103374 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103203 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-certs\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103374 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-credential-socket\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103374 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103276 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-data\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103541 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103409 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab242f-9d81-409d-abda-7577f121451c-istio-podinfo\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103541 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab242f-9d81-409d-abda-7577f121451c-istiod-ca-cert\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103627 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103554 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-certs\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103627 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103577 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-socket\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.103627 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.103620 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-credential-socket\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.104122 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.104096 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab242f-9d81-409d-abda-7577f121451c-istiod-ca-cert\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.105369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.105342 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-envoy\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.105668 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.105649 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab242f-9d81-409d-abda-7577f121451c-istio-podinfo\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.113502 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.113478 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-istio-token\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.114946 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.114924 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjr5\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-kube-api-access-cdjr5\") pod \"e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.182214 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.182194 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:20.463189 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.463165 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl"] Sep 10 18:00:20.464867 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:00:20.464827 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dab242f_9d81_409d_abda_7577f121451c.slice/crio-4fe553393b8e89a26f5821a82ec1580a2e454b868311e71a3e39d3bec22e2d79 WatchSource:0}: Error finding container 4fe553393b8e89a26f5821a82ec1580a2e454b868311e71a3e39d3bec22e2d79: Status 404 returned error can't find the container with id 4fe553393b8e89a26f5821a82ec1580a2e454b868311e71a3e39d3bec22e2d79 Sep 10 18:00:20.467113 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.467079 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:20.467195 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.467146 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:20.467195 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.467173 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:20.590723 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.590683 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" event={"ID":"3dab242f-9d81-409d-abda-7577f121451c","Type":"ContainerStarted","Data":"dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb"} Sep 10 18:00:20.590891 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.590745 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" event={"ID":"3dab242f-9d81-409d-abda-7577f121451c","Type":"ContainerStarted","Data":"4fe553393b8e89a26f5821a82ec1580a2e454b868311e71a3e39d3bec22e2d79"} Sep 10 18:00:20.623067 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:20.623007 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podStartSLOduration=1.622991228 podStartE2EDuration="1.622991228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:20.622106654 +0000 UTC m=+1149.503831796" watchObservedRunningTime="2026-09-10 18:00:20.622991228 +0000 UTC m=+1149.504716369" Sep 10 18:00:21.182999 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:21.182966 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:21.187776 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:21.187750 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:21.595926 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:21.595841 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:21.596666 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:21.596647 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:22.657432 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:22.657387 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:22.750093 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:22.750052 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:23.873558 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:23.873524 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:23.953465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:23.953430 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:24.458688 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.458651 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:24.556293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.556257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:24.658271 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.658211 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:24.681770 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.681738 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p"] Sep 10 18:00:24.685477 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.685461 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.689284 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.689258 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-09fbb5-dockercfg-cjrpn\"" Sep 10 18:00:24.689401 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.689336 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-09fbb5\"" Sep 10 18:00:24.697237 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.697202 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p"] Sep 10 18:00:24.741887 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.741797 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd1a003-4b12-4142-b660-6cc825685467-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.742051 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.741923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6775\" (UniqueName: \"kubernetes.io/projected/1bd1a003-4b12-4142-b660-6cc825685467-kube-api-access-j6775\") pod \"payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.842532 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.842484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6775\" (UniqueName: \"kubernetes.io/projected/1bd1a003-4b12-4142-b660-6cc825685467-kube-api-access-j6775\") pod \"payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.842707 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.842589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd1a003-4b12-4142-b660-6cc825685467-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.843266 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.843246 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd1a003-4b12-4142-b660-6cc825685467-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.852061 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.852039 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6775\" (UniqueName: \"kubernetes.io/projected/1bd1a003-4b12-4142-b660-6cc825685467-kube-api-access-j6775\") pod \"payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:24.997445 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:24.997362 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:25.270286 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:25.270260 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p"] Sep 10 18:00:25.271898 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:00:25.271854 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd1a003_4b12_4142_b660_6cc825685467.slice/crio-41f3257306dbc2bcd435079030ee6e7e72811069825b9baa9906b84caeaa0cad WatchSource:0}: Error finding container 41f3257306dbc2bcd435079030ee6e7e72811069825b9baa9906b84caeaa0cad: Status 404 returned error can't find the container with id 41f3257306dbc2bcd435079030ee6e7e72811069825b9baa9906b84caeaa0cad Sep 10 18:00:25.628049 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:25.627974 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:25.628049 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:25.628022 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" event={"ID":"1bd1a003-4b12-4142-b660-6cc825685467","Type":"ContainerStarted","Data":"71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66"} Sep 10 18:00:25.628049 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:25.628048 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" event={"ID":"1bd1a003-4b12-4142-b660-6cc825685467","Type":"ContainerStarted","Data":"41f3257306dbc2bcd435079030ee6e7e72811069825b9baa9906b84caeaa0cad"} Sep 10 18:00:25.651316 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:25.651269 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" podStartSLOduration=1.651255265 podStartE2EDuration="1.651255265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:25.649321082 +0000 UTC m=+1154.531046223" watchObservedRunningTime="2026-09-10 18:00:25.651255265 +0000 UTC m=+1154.532980406" Sep 10 18:00:33.495321 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:33.495282 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:36.633236 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:36.633191 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:00:40.718194 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:40.718158 2565 generic.go:356] "Generic (PLEG): container finished" podID="3dab242f-9d81-409d-abda-7577f121451c" containerID="dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb" exitCode=0 Sep 10 18:00:40.718604 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:40.718246 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" event={"ID":"3dab242f-9d81-409d-abda-7577f121451c","Type":"ContainerDied","Data":"dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb"} Sep 10 18:00:40.718697 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:40.718679 2565 scope.go:117] "RemoveContainer" containerID="dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb" Sep 10 18:00:40.719787 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:40.719755 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:40.719890 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:40.719829 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:40.719890 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:40.719863 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:41.725393 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:41.725359 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" event={"ID":"3dab242f-9d81-409d-abda-7577f121451c","Type":"ContainerStarted","Data":"63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d"} Sep 10 18:00:42.182625 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:42.182590 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:42.182837 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:42.182809 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Sep 10 18:00:42.182914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:42.182871 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Sep 10 18:00:43.182672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:43.182636 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Sep 10 18:00:43.183066 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:43.182688 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Sep 10 18:00:44.183559 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:44.183519 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Sep 10 18:00:44.183965 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:44.183575 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Sep 10 18:00:45.183171 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:45.183128 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Sep 10 18:00:45.183370 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:45.183197 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Sep 10 18:00:46.182916 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:46.182879 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Sep 10 18:00:46.183384 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:46.182933 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Sep 10 18:00:47.187635 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:47.187603 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:47.188027 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:47.187877 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:47.188488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:47.188470 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:00:47.489254 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:47.489152 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:54.153010 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:54.152979 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:54.919992 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:54.919961 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:56.678752 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:56.678711 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:56.967341 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:56.967257 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl"] Sep 10 18:00:56.970310 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:56.970287 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:56.973312 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:56.973290 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-be08bfb3-openshift-default-dockercfg-jzmch\"" Sep 10 18:00:56.984185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:56.984160 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl"] Sep 10 18:00:56.992627 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:56.992605 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:57.151182 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-podinfo\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151393 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151256 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-credential-socket\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151393 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151284 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aef89d-4395-45cb-837e-50fb97639fbf-istiod-ca-cert\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151393 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-envoy\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151393 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-socket\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151393 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-data\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151615 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-token\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151615 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-certs\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.151615 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.151535 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpg2\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-kube-api-access-nbpg2\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.252913 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.252828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-podinfo\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.252913 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.252908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-credential-socket\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253189 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.252941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aef89d-4395-45cb-837e-50fb97639fbf-istiod-ca-cert\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253189 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.252968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-envoy\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253189 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253017 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-socket\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253189 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253040 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-data\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253442 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253251 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-token\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253442 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-certs\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253442 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253359 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-credential-socket\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253442 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpg2\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-kube-api-access-nbpg2\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253442 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253423 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-socket\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253644 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253578 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-data\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253712 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253691 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-certs\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.253826 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.253810 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aef89d-4395-45cb-837e-50fb97639fbf-istiod-ca-cert\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.255381 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.255359 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-envoy\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.255596 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.255579 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-podinfo\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.262684 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.262659 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpg2\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-kube-api-access-nbpg2\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.262796 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.262754 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-token\") pod \"e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.280485 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.280464 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:57.326514 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.326483 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:57.562098 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.562072 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl"] Sep 10 18:00:57.563999 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:00:57.563968 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0aef89d_4395_45cb_837e_50fb97639fbf.slice/crio-ce175593e3029711fbfd3ed88780ece8303425d5126093641f53da6011c5656d WatchSource:0}: Error finding container ce175593e3029711fbfd3ed88780ece8303425d5126093641f53da6011c5656d: Status 404 returned error can't find the container with id ce175593e3029711fbfd3ed88780ece8303425d5126093641f53da6011c5656d Sep 10 18:00:57.566513 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.566472 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:57.566592 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.566570 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:57.566633 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.566616 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:00:57.813853 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.813769 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" event={"ID":"f0aef89d-4395-45cb-837e-50fb97639fbf","Type":"ContainerStarted","Data":"8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074"} Sep 10 18:00:57.813853 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.813834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" event={"ID":"f0aef89d-4395-45cb-837e-50fb97639fbf","Type":"ContainerStarted","Data":"ce175593e3029711fbfd3ed88780ece8303425d5126093641f53da6011c5656d"} Sep 10 18:00:57.837325 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:57.837279 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podStartSLOduration=1.8372632150000001 podStartE2EDuration="1.837263215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:57.834979604 +0000 UTC m=+1186.716704772" watchObservedRunningTime="2026-09-10 18:00:57.837263215 +0000 UTC m=+1186.718988355" Sep 10 18:00:58.281389 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:58.281360 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:58.286533 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:58.286510 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:58.819286 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:58.819254 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:58.820303 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:58.820286 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:00:59.714048 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:59.714017 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:00:59.809111 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:00:59.809076 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:00.877396 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:00.877358 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:00.930071 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:00.930038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:01.291124 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.291092 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:01.391471 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.391441 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:01.464112 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.464069 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:01.501552 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.501522 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw"] Sep 10 18:01:01.504281 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.504262 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.507397 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.507373 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-be08bfb3-dockercfg-x4w2m\"" Sep 10 18:01:01.507519 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.507386 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-be08bfb3\"" Sep 10 18:01:01.515906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.515885 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw"] Sep 10 18:01:01.695099 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.695068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd9693-7212-4840-851e-09a0a54ecb14-plugins-config-volume\") pod \"payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.695302 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.695106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk2lf\" (UniqueName: \"kubernetes.io/projected/39cd9693-7212-4840-851e-09a0a54ecb14-kube-api-access-qk2lf\") pod \"payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.796109 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.796071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd9693-7212-4840-851e-09a0a54ecb14-plugins-config-volume\") pod \"payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.796312 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.796114 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk2lf\" (UniqueName: \"kubernetes.io/projected/39cd9693-7212-4840-851e-09a0a54ecb14-kube-api-access-qk2lf\") pod \"payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.796819 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.796795 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd9693-7212-4840-851e-09a0a54ecb14-plugins-config-volume\") pod \"payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.805351 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.805333 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk2lf\" (UniqueName: \"kubernetes.io/projected/39cd9693-7212-4840-851e-09a0a54ecb14-kube-api-access-qk2lf\") pod \"payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:01.817122 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:01.817097 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:02.092536 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:02.092499 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw"] Sep 10 18:01:02.094742 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:01:02.094714 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cd9693_7212_4840_851e_09a0a54ecb14.slice/crio-0c48e53ef4291187abd01821c41fec063965850e571eab4d40de94a56bfd4634 WatchSource:0}: Error finding container 0c48e53ef4291187abd01821c41fec063965850e571eab4d40de94a56bfd4634: Status 404 returned error can't find the container with id 0c48e53ef4291187abd01821c41fec063965850e571eab4d40de94a56bfd4634 Sep 10 18:01:02.854533 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:02.854492 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" event={"ID":"39cd9693-7212-4840-851e-09a0a54ecb14","Type":"ContainerStarted","Data":"d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5"} Sep 10 18:01:02.854689 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:02.854565 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" event={"ID":"39cd9693-7212-4840-851e-09a0a54ecb14","Type":"ContainerStarted","Data":"0c48e53ef4291187abd01821c41fec063965850e571eab4d40de94a56bfd4634"} Sep 10 18:01:02.854689 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:02.854591 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:02.879041 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:02.878992 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" podStartSLOduration=1.878977362 podStartE2EDuration="1.878977362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:01:02.876504004 +0000 UTC m=+1191.758229146" watchObservedRunningTime="2026-09-10 18:01:02.878977362 +0000 UTC m=+1191.760702504" Sep 10 18:01:05.065194 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:05.065154 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:13.863977 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:13.863938 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:01:16.879608 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:16.879569 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:16.944194 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:16.944168 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:21.746297 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:21.746253 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:22.971900 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:22.971868 2565 generic.go:356] "Generic (PLEG): container finished" podID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerID="8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074" exitCode=0 Sep 10 18:01:22.972318 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:22.971941 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" event={"ID":"f0aef89d-4395-45cb-837e-50fb97639fbf","Type":"ContainerDied","Data":"8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074"} Sep 10 18:01:22.972370 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:22.972356 2565 scope.go:117] "RemoveContainer" containerID="8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074" Sep 10 18:01:22.973479 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:22.973447 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:01:22.973552 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:22.973522 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:01:22.973596 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:22.973557 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:01:23.981586 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:23.981550 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" event={"ID":"f0aef89d-4395-45cb-837e-50fb97639fbf","Type":"ContainerStarted","Data":"ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f"} Sep 10 18:01:24.281418 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:24.281332 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:01:24.282675 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:24.282642 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Sep 10 18:01:24.282823 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:24.282692 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Sep 10 18:01:24.667298 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:24.667177 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:25.281553 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:25.281516 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Sep 10 18:01:25.281939 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:25.281581 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Sep 10 18:01:26.281085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:26.281050 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Sep 10 18:01:26.281285 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:26.281100 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Sep 10 18:01:27.281544 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:27.281510 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Sep 10 18:01:27.282041 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:27.281567 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Sep 10 18:01:28.280860 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:28.280821 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Sep 10 18:01:28.281037 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:28.280886 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Sep 10 18:01:28.819339 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:28.819304 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:01:29.281314 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:29.281283 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Sep 10 18:01:29.281485 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:29.281349 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Sep 10 18:01:29.526574 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:29.526532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:01:30.285744 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:30.285714 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:01:30.286777 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:01:30.286760 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:02:03.924577 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:03.924497 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp"] Sep 10 18:02:03.926948 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:03.926931 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:03.929718 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:03.929697 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:02:03.929718 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:03.929699 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:02:03.935211 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:03.935186 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp"] Sep 10 18:02:04.013632 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:04.013603 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26dbb861-f7a4-486c-bdff-c8e319e0e74a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp\" (UID: \"26dbb861-f7a4-486c-bdff-c8e319e0e74a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:04.114469 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:04.114437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26dbb861-f7a4-486c-bdff-c8e319e0e74a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp\" (UID: \"26dbb861-f7a4-486c-bdff-c8e319e0e74a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:04.115016 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:04.114996 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26dbb861-f7a4-486c-bdff-c8e319e0e74a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp\" (UID: \"26dbb861-f7a4-486c-bdff-c8e319e0e74a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:04.239692 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:04.239632 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:04.515271 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:04.515083 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp"] Sep 10 18:02:04.518219 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:02:04.518191 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26dbb861_f7a4_486c_bdff_c8e319e0e74a.slice/crio-2db648c740a679608d2743eba0e0659396476819284d0513e03f238c4a0f5a42 WatchSource:0}: Error finding container 2db648c740a679608d2743eba0e0659396476819284d0513e03f238c4a0f5a42: Status 404 returned error can't find the container with id 2db648c740a679608d2743eba0e0659396476819284d0513e03f238c4a0f5a42 Sep 10 18:02:05.204502 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:05.204475 2565 generic.go:356] "Generic (PLEG): container finished" podID="26dbb861-f7a4-486c-bdff-c8e319e0e74a" containerID="08409a0f5688f7a368d769939a041d1030008c2d9b4d1960a16b9e5de2f7302d" exitCode=0 Sep 10 18:02:05.204813 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:05.204560 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" event={"ID":"26dbb861-f7a4-486c-bdff-c8e319e0e74a","Type":"ContainerDied","Data":"08409a0f5688f7a368d769939a041d1030008c2d9b4d1960a16b9e5de2f7302d"} Sep 10 18:02:05.204813 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:05.204624 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" event={"ID":"26dbb861-f7a4-486c-bdff-c8e319e0e74a","Type":"ContainerStarted","Data":"2db648c740a679608d2743eba0e0659396476819284d0513e03f238c4a0f5a42"} Sep 10 18:02:06.346394 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:06.346371 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:06.435452 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:06.435423 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26dbb861-f7a4-486c-bdff-c8e319e0e74a-maas-api-service-ca\") pod \"26dbb861-f7a4-486c-bdff-c8e319e0e74a\" (UID: \"26dbb861-f7a4-486c-bdff-c8e319e0e74a\") " Sep 10 18:02:06.435740 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:06.435719 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26dbb861-f7a4-486c-bdff-c8e319e0e74a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26dbb861-f7a4-486c-bdff-c8e319e0e74a" (UID: "26dbb861-f7a4-486c-bdff-c8e319e0e74a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:02:06.537150 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:06.537084 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26dbb861-f7a4-486c-bdff-c8e319e0e74a-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:02:07.225988 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:07.225959 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" Sep 10 18:02:07.226182 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:07.225960 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp" event={"ID":"26dbb861-f7a4-486c-bdff-c8e319e0e74a","Type":"ContainerDied","Data":"2db648c740a679608d2743eba0e0659396476819284d0513e03f238c4a0f5a42"} Sep 10 18:02:07.226182 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:07.226113 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2db648c740a679608d2743eba0e0659396476819284d0513e03f238c4a0f5a42" Sep 10 18:02:19.516502 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.516465 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:02:19.542380 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.542348 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw"] Sep 10 18:02:19.542633 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.542605 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" podUID="39cd9693-7212-4840-851e-09a0a54ecb14" containerName="payload-processing" containerID="cri-o://d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5" gracePeriod=30 Sep 10 18:02:19.716702 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.716665 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp"] Sep 10 18:02:19.720377 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.720352 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be08bfb3-p2cwp"] Sep 10 18:02:19.802268 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.802244 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:02:19.857176 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.857144 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd9693-7212-4840-851e-09a0a54ecb14-plugins-config-volume\") pod \"39cd9693-7212-4840-851e-09a0a54ecb14\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " Sep 10 18:02:19.857328 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.857307 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk2lf\" (UniqueName: \"kubernetes.io/projected/39cd9693-7212-4840-851e-09a0a54ecb14-kube-api-access-qk2lf\") pod \"39cd9693-7212-4840-851e-09a0a54ecb14\" (UID: \"39cd9693-7212-4840-851e-09a0a54ecb14\") " Sep 10 18:02:19.857571 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.857547 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cd9693-7212-4840-851e-09a0a54ecb14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39cd9693-7212-4840-851e-09a0a54ecb14" (UID: "39cd9693-7212-4840-851e-09a0a54ecb14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:02:19.857750 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.857726 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39cd9693-7212-4840-851e-09a0a54ecb14-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:02:19.859408 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.859376 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39cd9693-7212-4840-851e-09a0a54ecb14-kube-api-access-qk2lf" (OuterVolumeSpecName: "kube-api-access-qk2lf") pod "39cd9693-7212-4840-851e-09a0a54ecb14" (UID: "39cd9693-7212-4840-851e-09a0a54ecb14"). InnerVolumeSpecName "kube-api-access-qk2lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:02:19.958891 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:19.958849 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk2lf\" (UniqueName: \"kubernetes.io/projected/39cd9693-7212-4840-851e-09a0a54ecb14-kube-api-access-qk2lf\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:02:20.305601 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.305566 2565 generic.go:356] "Generic (PLEG): container finished" podID="39cd9693-7212-4840-851e-09a0a54ecb14" containerID="d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5" exitCode=0 Sep 10 18:02:20.305804 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.305610 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" Sep 10 18:02:20.305804 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.305661 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" event={"ID":"39cd9693-7212-4840-851e-09a0a54ecb14","Type":"ContainerDied","Data":"d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5"} Sep 10 18:02:20.305804 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.305720 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw" event={"ID":"39cd9693-7212-4840-851e-09a0a54ecb14","Type":"ContainerDied","Data":"0c48e53ef4291187abd01821c41fec063965850e571eab4d40de94a56bfd4634"} Sep 10 18:02:20.305804 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.305752 2565 scope.go:117] "RemoveContainer" containerID="d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5" Sep 10 18:02:20.315178 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.315161 2565 scope.go:117] "RemoveContainer" containerID="d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5" Sep 10 18:02:20.315466 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:02:20.315447 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5\": container with ID starting with d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5 not found: ID does not exist" containerID="d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5" Sep 10 18:02:20.315533 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.315475 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5"} err="failed to get container status \"d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5\": rpc error: code = NotFound desc = could not find container \"d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5\": container with ID starting with d95585f681d0b4c22d311a58411c31dd4efa951fe9b6573f090273d5320dc6e5 not found: ID does not exist" Sep 10 18:02:20.331312 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.331288 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw"] Sep 10 18:02:20.333614 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.333596 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be08bfb3-66b9ddc5df-vgcpw"] Sep 10 18:02:20.641882 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:20.641779 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:02:21.623636 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:21.623602 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26dbb861-f7a4-486c-bdff-c8e319e0e74a" path="/var/lib/kubelet/pods/26dbb861-f7a4-486c-bdff-c8e319e0e74a/volumes" Sep 10 18:02:21.623979 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:21.623964 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39cd9693-7212-4840-851e-09a0a54ecb14" path="/var/lib/kubelet/pods/39cd9693-7212-4840-851e-09a0a54ecb14/volumes" Sep 10 18:02:22.837096 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:22.837060 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:02:27.851412 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:27.851376 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:02:28.990500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:02:28.990416 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:04.318501 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:04.318468 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:05.614482 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:05.614453 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:11.465011 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:11.464973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:11.523974 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:11.523937 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:11.717805 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:11.717713 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl"] Sep 10 18:03:11.718000 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:11.717920 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" containerID="cri-o://ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f" gracePeriod=30 Sep 10 18:03:11.737120 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:11.737085 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:13.000538 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:13.000503 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:13.100548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:13.100515 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:13.201602 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:13.201570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:13.881104 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:13.881074 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:13.973919 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:13.973889 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:14.076350 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:14.076313 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:15.288107 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:15.288058 2565 patch_prober.go:28] interesting pod/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 18:03:15.288528 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:15.288119 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 18:03:15.298278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:15.298254 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:16.968236 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:16.968201 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:03:17.087200 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087139 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-envoy\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087200 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087183 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-data\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087425 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087207 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-certs\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087425 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087249 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-podinfo\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087425 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087273 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aef89d-4395-45cb-837e-50fb97639fbf-istiod-ca-cert\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087425 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087388 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbpg2\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-kube-api-access-nbpg2\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087443 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-credential-socket\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087479 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:03:17.087645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087486 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:03:17.087645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087541 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-token\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087589 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-socket\") pod \"f0aef89d-4395-45cb-837e-50fb97639fbf\" (UID: \"f0aef89d-4395-45cb-837e-50fb97639fbf\") " Sep 10 18:03:17.087906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087629 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0aef89d-4395-45cb-837e-50fb97639fbf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:03:17.087906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087700 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:03:17.087906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.087891 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:03:17.088085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.088010 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.088085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.088027 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.088085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.088041 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.088085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.088054 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.088085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.088065 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0aef89d-4395-45cb-837e-50fb97639fbf-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.089383 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.089361 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:03:17.089495 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.089472 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:03:17.089681 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.089656 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-kube-api-access-nbpg2" (OuterVolumeSpecName: "kube-api-access-nbpg2") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "kube-api-access-nbpg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:03:17.089773 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.089674 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0aef89d-4395-45cb-837e-50fb97639fbf" (UID: "f0aef89d-4395-45cb-837e-50fb97639fbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:03:17.189082 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.189058 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbpg2\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-kube-api-access-nbpg2\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.189082 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.189081 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.189204 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.189090 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.189204 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.189098 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0aef89d-4395-45cb-837e-50fb97639fbf-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:03:17.634843 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.634819 2565 generic.go:356] "Generic (PLEG): container finished" podID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerID="ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f" exitCode=0 Sep 10 18:03:17.634959 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.634863 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" Sep 10 18:03:17.634959 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.634904 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" event={"ID":"f0aef89d-4395-45cb-837e-50fb97639fbf","Type":"ContainerDied","Data":"ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f"} Sep 10 18:03:17.635102 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.634962 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl" event={"ID":"f0aef89d-4395-45cb-837e-50fb97639fbf","Type":"ContainerDied","Data":"ce175593e3029711fbfd3ed88780ece8303425d5126093641f53da6011c5656d"} Sep 10 18:03:17.635102 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.634994 2565 scope.go:117] "RemoveContainer" containerID="ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f" Sep 10 18:03:17.644385 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.644366 2565 scope.go:117] "RemoveContainer" containerID="8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074" Sep 10 18:03:17.652264 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.652248 2565 scope.go:117] "RemoveContainer" containerID="ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f" Sep 10 18:03:17.652534 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:03:17.652517 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f\": container with ID starting with ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f not found: ID does not exist" containerID="ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f" Sep 10 18:03:17.652575 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.652541 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f"} err="failed to get container status \"ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f\": rpc error: code = NotFound desc = could not find container \"ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f\": container with ID starting with ce00046141c9b0fb66156afc2bb18845109690edb64b2a3907267b5dbe8a606f not found: ID does not exist" Sep 10 18:03:17.652575 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.652557 2565 scope.go:117] "RemoveContainer" containerID="8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074" Sep 10 18:03:17.652813 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:03:17.652795 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074\": container with ID starting with 8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074 not found: ID does not exist" containerID="8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074" Sep 10 18:03:17.652850 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.652820 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074"} err="failed to get container status \"8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074\": rpc error: code = NotFound desc = could not find container \"8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074\": container with ID starting with 8651051b5a50338d5a9f81eab3fc60b88169650f40ac272156272413d665b074 not found: ID does not exist" Sep 10 18:03:17.670023 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.670000 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl"] Sep 10 18:03:17.680043 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:17.680023 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-be08bfb3-openshift-default-79c978f6fb-vprnl"] Sep 10 18:03:19.623203 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:19.623169 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" path="/var/lib/kubelet/pods/f0aef89d-4395-45cb-837e-50fb97639fbf/volumes" Sep 10 18:03:21.333806 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:21.333764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:21.400729 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:21.400690 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:21.998327 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:21.998293 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:23.399483 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:23.399447 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:24.478913 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:24.478883 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:27.061977 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:27.061938 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:28.884099 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:28.884060 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:28.942675 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:28.942646 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:29.008364 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:29.008337 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:29.110978 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:29.110951 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:31.185614 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:31.185584 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:39.613881 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:39.613795 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:39.839840 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:39.839798 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:03:48.551430 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:03:48.551389 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:00.802910 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:00.802870 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:07.012491 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.012460 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:07.074915 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.074883 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:07.361530 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.361453 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:07.667484 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667405 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh"] Sep 10 18:04:07.667848 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667834 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26dbb861-f7a4-486c-bdff-c8e319e0e74a" containerName="revoke-keys" Sep 10 18:04:07.667892 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667850 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26dbb861-f7a4-486c-bdff-c8e319e0e74a" containerName="revoke-keys" Sep 10 18:04:07.667892 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667864 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cd9693-7212-4840-851e-09a0a54ecb14" containerName="payload-processing" Sep 10 18:04:07.667892 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667871 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cd9693-7212-4840-851e-09a0a54ecb14" containerName="payload-processing" Sep 10 18:04:07.667892 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667890 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" Sep 10 18:04:07.668032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667896 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" Sep 10 18:04:07.668032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667905 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" Sep 10 18:04:07.668032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667910 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" Sep 10 18:04:07.668032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667979 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cd9693-7212-4840-851e-09a0a54ecb14" containerName="payload-processing" Sep 10 18:04:07.668032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667990 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" Sep 10 18:04:07.668032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.667997 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26dbb861-f7a4-486c-bdff-c8e319e0e74a" containerName="revoke-keys" Sep 10 18:04:07.670306 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.670282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.673379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.673360 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b38800-openshift-default-dockercfg-mpc24\"" Sep 10 18:04:07.687387 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.687361 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh"] Sep 10 18:04:07.705250 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.705209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:07.736870 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.736844 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tccn9\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-kube-api-access-tccn9\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.736993 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.736890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-socket\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.736993 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.736927 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-credential-socket\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.737099 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.736973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-envoy\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.737150 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.737128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-data\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.737188 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.737161 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istiod-ca-cert\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.737267 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.737201 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-podinfo\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.737267 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.737251 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-token\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.737367 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.737334 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-certs\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.837890 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.837855 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-socket\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838053 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.837899 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-credential-socket\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838053 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-envoy\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838304 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838186 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-data\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838304 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838269 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istiod-ca-cert\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838522 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838326 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-podinfo\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838522 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-socket\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838522 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838383 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-token\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838522 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-certs\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838522 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tccn9\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-kube-api-access-tccn9\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838785 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838572 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-data\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.838785 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.838712 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-credential-socket\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.839122 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.839075 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istiod-ca-cert\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.839252 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.839196 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-certs\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.841025 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.841005 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-envoy\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.841170 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.841152 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-podinfo\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.849050 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.849025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-token\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.849270 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.849249 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tccn9\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-kube-api-access-tccn9\") pod \"e2e-api-a-b38800-openshift-default-5bb5985786-nptlh\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:07.889377 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.889342 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:07.981492 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:07.981456 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:08.269937 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.269861 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh"] Sep 10 18:04:08.273856 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:04:08.273825 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9a08da_727d_4c92_9cfd_1caaaf80cef3.slice/crio-380df6cfbf7228a1e360324a24781d9d4273df6f06408d42917d77a5f16c4249 WatchSource:0}: Error finding container 380df6cfbf7228a1e360324a24781d9d4273df6f06408d42917d77a5f16c4249: Status 404 returned error can't find the container with id 380df6cfbf7228a1e360324a24781d9d4273df6f06408d42917d77a5f16c4249 Sep 10 18:04:08.276207 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.276174 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:08.276395 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.276273 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:08.276395 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.276320 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:08.923135 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.923096 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" event={"ID":"7f9a08da-727d-4c92-9cfd-1caaaf80cef3","Type":"ContainerStarted","Data":"7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5"} Sep 10 18:04:08.923321 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.923153 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" event={"ID":"7f9a08da-727d-4c92-9cfd-1caaaf80cef3","Type":"ContainerStarted","Data":"380df6cfbf7228a1e360324a24781d9d4273df6f06408d42917d77a5f16c4249"} Sep 10 18:04:08.951330 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.951281 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" podStartSLOduration=1.9512641739999999 podStartE2EDuration="1.951264174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:08.948332591 +0000 UTC m=+1377.830057734" watchObservedRunningTime="2026-09-10 18:04:08.951264174 +0000 UTC m=+1377.832989356" Sep 10 18:04:08.982275 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:08.982249 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:09.986929 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:09.986893 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:10.888797 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:10.888767 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:10.934729 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:10.934706 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:10.936208 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:10.936184 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:10.975197 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:10.975163 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:11.752361 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:11.752330 2565 scope.go:117] "RemoveContainer" containerID="f486aa104bf52aa0899b25aaffaea4692300f71175928959627ae3b0f3ddf2da" Sep 10 18:04:12.262573 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:12.262530 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:12.355645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:12.355612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:12.512215 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:12.512180 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:12.900417 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:12.900383 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:12.992465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:12.992435 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:13.095362 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.095333 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l"] Sep 10 18:04:13.095899 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.095883 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0aef89d-4395-45cb-837e-50fb97639fbf" containerName="istio-proxy" Sep 10 18:04:13.098750 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.098731 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.101471 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.101450 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b38800-dockercfg-t2tvg\"" Sep 10 18:04:13.101471 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.101461 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b38800\"" Sep 10 18:04:13.115800 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.115777 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l"] Sep 10 18:04:13.191075 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.191037 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjk9l\" (UniqueName: \"kubernetes.io/projected/ddb444af-db87-4d4d-b2ad-1e49c43d9314-kube-api-access-kjk9l\") pod \"payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.191245 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.191174 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb444af-db87-4d4d-b2ad-1e49c43d9314-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.291806 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.291777 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb444af-db87-4d4d-b2ad-1e49c43d9314-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.291949 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.291850 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjk9l\" (UniqueName: \"kubernetes.io/projected/ddb444af-db87-4d4d-b2ad-1e49c43d9314-kube-api-access-kjk9l\") pod \"payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.292485 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.292468 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb444af-db87-4d4d-b2ad-1e49c43d9314-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.303006 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.302985 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjk9l\" (UniqueName: \"kubernetes.io/projected/ddb444af-db87-4d4d-b2ad-1e49c43d9314-kube-api-access-kjk9l\") pod \"payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.409057 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.409031 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.683198 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.683172 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l"] Sep 10 18:04:13.684376 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:04:13.684347 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb444af_db87_4d4d_b2ad_1e49c43d9314.slice/crio-f78107b906f7790408943d6615621d51e112b3e53d31a024610cd76ad2bfac97 WatchSource:0}: Error finding container f78107b906f7790408943d6615621d51e112b3e53d31a024610cd76ad2bfac97: Status 404 returned error can't find the container with id f78107b906f7790408943d6615621d51e112b3e53d31a024610cd76ad2bfac97 Sep 10 18:04:13.952044 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.952003 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" event={"ID":"ddb444af-db87-4d4d-b2ad-1e49c43d9314","Type":"ContainerStarted","Data":"75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0"} Sep 10 18:04:13.952428 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.952066 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" event={"ID":"ddb444af-db87-4d4d-b2ad-1e49c43d9314","Type":"ContainerStarted","Data":"f78107b906f7790408943d6615621d51e112b3e53d31a024610cd76ad2bfac97"} Sep 10 18:04:13.952428 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.952131 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:13.974973 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:13.974926 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" podStartSLOduration=0.974912194 podStartE2EDuration="974.912194ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:13.973726193 +0000 UTC m=+1382.855451334" watchObservedRunningTime="2026-09-10 18:04:13.974912194 +0000 UTC m=+1382.856637335" Sep 10 18:04:24.960469 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:24.960426 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:04:32.057816 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:32.057781 2565 generic.go:356] "Generic (PLEG): container finished" podID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerID="7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5" exitCode=0 Sep 10 18:04:32.058271 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:32.057851 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" event={"ID":"7f9a08da-727d-4c92-9cfd-1caaaf80cef3","Type":"ContainerDied","Data":"7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5"} Sep 10 18:04:32.058271 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:32.058268 2565 scope.go:117] "RemoveContainer" containerID="7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5" Sep 10 18:04:32.059338 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:32.059306 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:32.059432 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:32.059375 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:32.059432 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:32.059406 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:33.070696 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:33.070659 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" event={"ID":"7f9a08da-727d-4c92-9cfd-1caaaf80cef3","Type":"ContainerStarted","Data":"1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1"} Sep 10 18:04:33.981813 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:33.981773 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:33.981995 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:33.981898 2565 patch_prober.go:28] interesting pod/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Sep 10 18:04:33.981995 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:33.981943 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Sep 10 18:04:34.380278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:34.380167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:34.982702 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:34.982669 2565 patch_prober.go:28] interesting pod/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Sep 10 18:04:34.982875 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:34.982721 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Sep 10 18:04:35.448269 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:35.448209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:35.824508 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:35.824432 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:35.982508 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:35.982462 2565 patch_prober.go:28] interesting pod/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Sep 10 18:04:35.982656 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:35.982525 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Sep 10 18:04:36.256951 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.256921 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:36.586473 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.586366 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t"] Sep 10 18:04:36.589488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.589465 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.593046 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.593026 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d439f7-openshift-default-dockercfg-7b9db\"" Sep 10 18:04:36.608892 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.608868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-certs\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609124 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609097 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7m54\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-kube-api-access-h7m54\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609250 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609142 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-data\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609250 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609194 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-podinfo\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609373 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-socket\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609373 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-token\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609373 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609363 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-credential-socket\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609529 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609472 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-envoy\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.609529 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.609517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e43be7e3-3df7-4696-925f-be3ea041f9a7-istiod-ca-cert\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.612389 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.612366 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t"] Sep 10 18:04:36.616463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.616437 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:36.710722 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.710682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-socket\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.711401 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.711373 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-socket\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.711579 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.711430 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-token\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.711733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.711718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-credential-socket\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.711903 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.711889 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-envoy\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.712036 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712004 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e43be7e3-3df7-4696-925f-be3ea041f9a7-istiod-ca-cert\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.712447 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712422 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-credential-socket\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.712545 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712502 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-certs\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.712613 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7m54\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-kube-api-access-h7m54\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.712613 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-data\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.712728 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712617 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-podinfo\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.713278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712910 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-certs\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.713278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.712973 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e43be7e3-3df7-4696-925f-be3ea041f9a7-istiod-ca-cert\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.713431 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.713322 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-data\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.718360 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.718336 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-podinfo\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.719619 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.719595 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-envoy\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.733026 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.732996 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-token\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.733813 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.733790 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7m54\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-kube-api-access-h7m54\") pod \"e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.900218 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.900130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:36.986672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.986639 2565 patch_prober.go:28] interesting pod/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Sep 10 18:04:36.986825 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:36.986704 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Sep 10 18:04:37.211135 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:37.211110 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t"] Sep 10 18:04:37.213078 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:04:37.213052 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43be7e3_3df7_4696_925f_be3ea041f9a7.slice/crio-eb12276652f28ccdca8591af26b28f817441f358e6bb8fc7e04def6b3f8a8f9b WatchSource:0}: Error finding container eb12276652f28ccdca8591af26b28f817441f358e6bb8fc7e04def6b3f8a8f9b: Status 404 returned error can't find the container with id eb12276652f28ccdca8591af26b28f817441f358e6bb8fc7e04def6b3f8a8f9b Sep 10 18:04:37.215398 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:37.215364 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:37.215463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:37.215433 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:37.215500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:37.215461 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:37.216841 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:37.216815 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:38.104840 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.104802 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" event={"ID":"e43be7e3-3df7-4696-925f-be3ea041f9a7","Type":"ContainerStarted","Data":"b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11"} Sep 10 18:04:38.105264 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.104870 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" event={"ID":"e43be7e3-3df7-4696-925f-be3ea041f9a7","Type":"ContainerStarted","Data":"eb12276652f28ccdca8591af26b28f817441f358e6bb8fc7e04def6b3f8a8f9b"} Sep 10 18:04:38.138411 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.138360 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podStartSLOduration=2.138344839 podStartE2EDuration="2.138344839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:38.134783881 +0000 UTC m=+1407.016509048" watchObservedRunningTime="2026-09-10 18:04:38.138344839 +0000 UTC m=+1407.020069979" Sep 10 18:04:38.165644 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.165616 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:38.165953 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.165932 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:38.166898 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.166877 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:04:38.901201 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.901171 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:38.906145 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:38.906122 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:39.110756 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:39.110726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:39.111481 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:39.111459 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:40.253296 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:40.253263 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:40.334815 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:40.334781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:41.437544 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:41.437506 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:41.534543 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:41.534508 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:41.813547 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:41.813441 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:42.090453 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.090383 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:42.214360 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.214313 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:42.317018 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.316982 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr"] Sep 10 18:04:42.320374 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.320357 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.323980 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.323954 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d439f7-dockercfg-lkgkq\"" Sep 10 18:04:42.324464 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.324448 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d439f7\"" Sep 10 18:04:42.341713 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.341659 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr"] Sep 10 18:04:42.367682 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.367658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfhhx\" (UniqueName: \"kubernetes.io/projected/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-kube-api-access-cfhhx\") pod \"payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.367833 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.367814 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.469412 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.469381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfhhx\" (UniqueName: \"kubernetes.io/projected/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-kube-api-access-cfhhx\") pod \"payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.469790 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.469546 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.470208 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.470189 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.479682 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.479657 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfhhx\" (UniqueName: \"kubernetes.io/projected/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-kube-api-access-cfhhx\") pod \"payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.631940 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.631876 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:42.910278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:42.910253 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr"] Sep 10 18:04:42.912122 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:04:42.912096 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4641a2e_bcae_4c4e_94ce_71a9e7d9e6c1.slice/crio-08db2de3b2616649fad23519a4dc8b6e9e0904ec0bd81d1c23293da47bf14618 WatchSource:0}: Error finding container 08db2de3b2616649fad23519a4dc8b6e9e0904ec0bd81d1c23293da47bf14618: Status 404 returned error can't find the container with id 08db2de3b2616649fad23519a4dc8b6e9e0904ec0bd81d1c23293da47bf14618 Sep 10 18:04:43.133930 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:43.133889 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" event={"ID":"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1","Type":"ContainerStarted","Data":"26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5"} Sep 10 18:04:43.134089 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:43.133956 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" event={"ID":"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1","Type":"ContainerStarted","Data":"08db2de3b2616649fad23519a4dc8b6e9e0904ec0bd81d1c23293da47bf14618"} Sep 10 18:04:43.134134 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:43.134110 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:43.154293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:43.154239 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" podStartSLOduration=1.154216372 podStartE2EDuration="1.154216372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:43.151817312 +0000 UTC m=+1412.033542453" watchObservedRunningTime="2026-09-10 18:04:43.154216372 +0000 UTC m=+1412.035941513" Sep 10 18:04:46.630293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:46.630260 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:52.350715 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:52.350644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:04:54.140783 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:54.140748 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:04:58.226122 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:58.226086 2565 generic.go:356] "Generic (PLEG): container finished" podID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerID="b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11" exitCode=0 Sep 10 18:04:58.226557 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:58.226162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" event={"ID":"e43be7e3-3df7-4696-925f-be3ea041f9a7","Type":"ContainerDied","Data":"b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11"} Sep 10 18:04:58.226724 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:58.226706 2565 scope.go:117] "RemoveContainer" containerID="b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11" Sep 10 18:04:58.227897 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:58.227867 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:58.227960 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:58.227940 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:58.228007 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:58.227995 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:04:59.233468 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:59.233430 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" event={"ID":"e43be7e3-3df7-4696-925f-be3ea041f9a7","Type":"ContainerStarted","Data":"c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8"} Sep 10 18:04:59.900897 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:59.900865 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:04:59.901090 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:59.901063 2565 patch_prober.go:28] interesting pod/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 10 18:04:59.901139 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:04:59.901106 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 10 18:05:00.900887 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:00.900848 2565 patch_prober.go:28] interesting pod/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 10 18:05:00.901287 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:00.900906 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 10 18:05:01.901326 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:01.901288 2565 patch_prober.go:28] interesting pod/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 10 18:05:01.901703 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:01.901358 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 10 18:05:02.754024 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:02.753979 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:02.901209 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:02.901170 2565 patch_prober.go:28] interesting pod/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 10 18:05:02.901406 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:02.901253 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 10 18:05:03.901381 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:03.901293 2565 patch_prober.go:28] interesting pod/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 10 18:05:03.901381 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:03.901366 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 10 18:05:04.904429 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:04.904396 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:05:04.904824 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:04.904672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:05:04.905398 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:04.905376 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:05:05.109721 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:05.109688 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:07.977505 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:07.977473 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb"] Sep 10 18:05:07.980215 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:07.980191 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:07.983087 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:07.983064 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:05:07.983256 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:07.983240 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:05:08.000495 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.000380 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb"] Sep 10 18:05:08.042611 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.042579 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3858e9c-09a2-49c6-9a67-6e83a38ada4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b38800-nqxmb\" (UID: \"b3858e9c-09a2-49c6-9a67-6e83a38ada4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:08.090195 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.090158 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:08.144310 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.144277 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3858e9c-09a2-49c6-9a67-6e83a38ada4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b38800-nqxmb\" (UID: \"b3858e9c-09a2-49c6-9a67-6e83a38ada4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:08.144852 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.144832 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3858e9c-09a2-49c6-9a67-6e83a38ada4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b38800-nqxmb\" (UID: \"b3858e9c-09a2-49c6-9a67-6e83a38ada4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:08.293096 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.293014 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:08.570842 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:08.570817 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb"] Sep 10 18:05:08.572714 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:05:08.572687 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3858e9c_09a2_49c6_9a67_6e83a38ada4d.slice/crio-ecb8f72ae98ad3260f4438ca15db3fcb70699526713f9c7b10157bf75066fa9e WatchSource:0}: Error finding container ecb8f72ae98ad3260f4438ca15db3fcb70699526713f9c7b10157bf75066fa9e: Status 404 returned error can't find the container with id ecb8f72ae98ad3260f4438ca15db3fcb70699526713f9c7b10157bf75066fa9e Sep 10 18:05:09.300111 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:09.300079 2565 generic.go:356] "Generic (PLEG): container finished" podID="b3858e9c-09a2-49c6-9a67-6e83a38ada4d" containerID="07db3464d60bbfb87ebd57df4c022b98a03c389173de9297cc28d8854368d4d7" exitCode=0 Sep 10 18:05:09.300483 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:09.300162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" event={"ID":"b3858e9c-09a2-49c6-9a67-6e83a38ada4d","Type":"ContainerDied","Data":"07db3464d60bbfb87ebd57df4c022b98a03c389173de9297cc28d8854368d4d7"} Sep 10 18:05:09.300483 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:09.300221 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" event={"ID":"b3858e9c-09a2-49c6-9a67-6e83a38ada4d","Type":"ContainerStarted","Data":"ecb8f72ae98ad3260f4438ca15db3fcb70699526713f9c7b10157bf75066fa9e"} Sep 10 18:05:10.443012 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:10.442991 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:10.470065 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:10.470039 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3858e9c-09a2-49c6-9a67-6e83a38ada4d-maas-api-service-ca\") pod \"b3858e9c-09a2-49c6-9a67-6e83a38ada4d\" (UID: \"b3858e9c-09a2-49c6-9a67-6e83a38ada4d\") " Sep 10 18:05:10.470465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:10.470440 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3858e9c-09a2-49c6-9a67-6e83a38ada4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3858e9c-09a2-49c6-9a67-6e83a38ada4d" (UID: "b3858e9c-09a2-49c6-9a67-6e83a38ada4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:10.571737 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:10.571652 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3858e9c-09a2-49c6-9a67-6e83a38ada4d-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:11.311704 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:11.311675 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" Sep 10 18:05:11.311880 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:11.311710 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb" event={"ID":"b3858e9c-09a2-49c6-9a67-6e83a38ada4d","Type":"ContainerDied","Data":"ecb8f72ae98ad3260f4438ca15db3fcb70699526713f9c7b10157bf75066fa9e"} Sep 10 18:05:11.311880 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:11.311756 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecb8f72ae98ad3260f4438ca15db3fcb70699526713f9c7b10157bf75066fa9e" Sep 10 18:05:11.777282 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:11.777218 2565 scope.go:117] "RemoveContainer" containerID="a6e604ce66e432a8cdb8a6dc0467b8626f9c8a3f5ba11ff7570017157b1664ca" Sep 10 18:05:15.436801 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:15.436764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:17.561035 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:17.561000 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:23.293161 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.293130 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:23.391060 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.391031 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l"] Sep 10 18:05:23.391672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.391634 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" podUID="ddb444af-db87-4d4d-b2ad-1e49c43d9314" containerName="payload-pre-processing" containerID="cri-o://75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0" gracePeriod=30 Sep 10 18:05:23.437799 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.437767 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:23.523604 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.523577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:23.656883 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.656859 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:05:23.667490 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.667463 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:23.722829 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.722804 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb"] Sep 10 18:05:23.726723 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.726699 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b38800-nqxmb"] Sep 10 18:05:23.736445 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.736418 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:23.811499 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.811466 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb444af-db87-4d4d-b2ad-1e49c43d9314-plugins-config-volume\") pod \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " Sep 10 18:05:23.811694 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.811605 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjk9l\" (UniqueName: \"kubernetes.io/projected/ddb444af-db87-4d4d-b2ad-1e49c43d9314-kube-api-access-kjk9l\") pod \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\" (UID: \"ddb444af-db87-4d4d-b2ad-1e49c43d9314\") " Sep 10 18:05:23.811876 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.811851 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb444af-db87-4d4d-b2ad-1e49c43d9314-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddb444af-db87-4d4d-b2ad-1e49c43d9314" (UID: "ddb444af-db87-4d4d-b2ad-1e49c43d9314"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:23.812048 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.812031 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb444af-db87-4d4d-b2ad-1e49c43d9314-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:23.813668 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.813647 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb444af-db87-4d4d-b2ad-1e49c43d9314-kube-api-access-kjk9l" (OuterVolumeSpecName: "kube-api-access-kjk9l") pod "ddb444af-db87-4d4d-b2ad-1e49c43d9314" (UID: "ddb444af-db87-4d4d-b2ad-1e49c43d9314"). InnerVolumeSpecName "kube-api-access-kjk9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:05:23.913374 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:23.913345 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjk9l\" (UniqueName: \"kubernetes.io/projected/ddb444af-db87-4d4d-b2ad-1e49c43d9314-kube-api-access-kjk9l\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:24.095934 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.095867 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:24.390765 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.390691 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddb444af-db87-4d4d-b2ad-1e49c43d9314" containerID="75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0" exitCode=0 Sep 10 18:05:24.390765 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.390732 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" Sep 10 18:05:24.391153 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.390779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" event={"ID":"ddb444af-db87-4d4d-b2ad-1e49c43d9314","Type":"ContainerDied","Data":"75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0"} Sep 10 18:05:24.391153 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.390835 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l" event={"ID":"ddb444af-db87-4d4d-b2ad-1e49c43d9314","Type":"ContainerDied","Data":"f78107b906f7790408943d6615621d51e112b3e53d31a024610cd76ad2bfac97"} Sep 10 18:05:24.391153 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.390862 2565 scope.go:117] "RemoveContainer" containerID="75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0" Sep 10 18:05:24.400265 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.400104 2565 scope.go:117] "RemoveContainer" containerID="75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0" Sep 10 18:05:24.400406 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:05:24.400388 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0\": container with ID starting with 75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0 not found: ID does not exist" containerID="75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0" Sep 10 18:05:24.400450 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.400414 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0"} err="failed to get container status \"75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0\": rpc error: code = NotFound desc = could not find container \"75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0\": container with ID starting with 75546fcf170a0d0910c59b0065933858056a124d6a71d96e510f98ba255f8db0 not found: ID does not exist" Sep 10 18:05:24.420553 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.420527 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l"] Sep 10 18:05:24.427184 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:24.427163 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b38800-d99c4f5cd-sx82l"] Sep 10 18:05:25.623619 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:25.623585 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3858e9c-09a2-49c6-9a67-6e83a38ada4d" path="/var/lib/kubelet/pods/b3858e9c-09a2-49c6-9a67-6e83a38ada4d/volumes" Sep 10 18:05:25.624002 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:25.623980 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddb444af-db87-4d4d-b2ad-1e49c43d9314" path="/var/lib/kubelet/pods/ddb444af-db87-4d4d-b2ad-1e49c43d9314/volumes" Sep 10 18:05:30.687625 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:30.687576 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:30.751453 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:30.751415 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:30.939924 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:30.939844 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh"] Sep 10 18:05:30.940128 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:30.940102 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" containerID="cri-o://1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1" gracePeriod=30 Sep 10 18:05:30.955675 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:30.955649 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:31.327378 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327300 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994"] Sep 10 18:05:31.327751 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327739 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb444af-db87-4d4d-b2ad-1e49c43d9314" containerName="payload-pre-processing" Sep 10 18:05:31.327796 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327753 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb444af-db87-4d4d-b2ad-1e49c43d9314" containerName="payload-pre-processing" Sep 10 18:05:31.327796 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327763 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3858e9c-09a2-49c6-9a67-6e83a38ada4d" containerName="revoke-keys" Sep 10 18:05:31.327796 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327768 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3858e9c-09a2-49c6-9a67-6e83a38ada4d" containerName="revoke-keys" Sep 10 18:05:31.327894 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327835 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb444af-db87-4d4d-b2ad-1e49c43d9314" containerName="payload-pre-processing" Sep 10 18:05:31.327894 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.327845 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3858e9c-09a2-49c6-9a67-6e83a38ada4d" containerName="revoke-keys" Sep 10 18:05:31.332158 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.332143 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:31.337855 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.337836 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:05:31.338006 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.337988 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:05:31.352033 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.352007 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994"] Sep 10 18:05:31.485163 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.485129 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d9291f-5789-49f7-8596-e3ad311ab0d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d439f7-c8994\" (UID: \"f7d9291f-5789-49f7-8596-e3ad311ab0d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:31.503761 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.503732 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:31.586652 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.586566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d9291f-5789-49f7-8596-e3ad311ab0d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d439f7-c8994\" (UID: \"f7d9291f-5789-49f7-8596-e3ad311ab0d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:31.587201 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.587179 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d9291f-5789-49f7-8596-e3ad311ab0d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d439f7-c8994\" (UID: \"f7d9291f-5789-49f7-8596-e3ad311ab0d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:31.642484 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.642459 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:31.917859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:31.917835 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994"] Sep 10 18:05:31.919460 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:05:31.919422 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d9291f_5789_49f7_8596_e3ad311ab0d5.slice/crio-6f64e7a55cbd18e2b30c7c5baa00e802b5a0bd060ceffd8577efcb1a3afc6d25 WatchSource:0}: Error finding container 6f64e7a55cbd18e2b30c7c5baa00e802b5a0bd060ceffd8577efcb1a3afc6d25: Status 404 returned error can't find the container with id 6f64e7a55cbd18e2b30c7c5baa00e802b5a0bd060ceffd8577efcb1a3afc6d25 Sep 10 18:05:32.437580 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:32.437541 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" event={"ID":"f7d9291f-5789-49f7-8596-e3ad311ab0d5","Type":"ContainerStarted","Data":"517f94e8f53603307f4bc51c505dd1734b63ee724a58a79c39c9913f019cdc9e"} Sep 10 18:05:32.437737 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:32.437591 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" event={"ID":"f7d9291f-5789-49f7-8596-e3ad311ab0d5","Type":"ContainerStarted","Data":"6f64e7a55cbd18e2b30c7c5baa00e802b5a0bd060ceffd8577efcb1a3afc6d25"} Sep 10 18:05:32.458497 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:32.458450 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" podStartSLOduration=1.458434598 podStartE2EDuration="1.458434598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:05:32.457797911 +0000 UTC m=+1461.339523054" watchObservedRunningTime="2026-09-10 18:05:32.458434598 +0000 UTC m=+1461.340159776" Sep 10 18:05:33.444859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:33.444825 2565 generic.go:356] "Generic (PLEG): container finished" podID="f7d9291f-5789-49f7-8596-e3ad311ab0d5" containerID="517f94e8f53603307f4bc51c505dd1734b63ee724a58a79c39c9913f019cdc9e" exitCode=0 Sep 10 18:05:33.445258 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:33.444881 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" event={"ID":"f7d9291f-5789-49f7-8596-e3ad311ab0d5","Type":"ContainerDied","Data":"517f94e8f53603307f4bc51c505dd1734b63ee724a58a79c39c9913f019cdc9e"} Sep 10 18:05:34.588312 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:34.588284 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:34.715599 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:34.715506 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d9291f-5789-49f7-8596-e3ad311ab0d5-maas-api-service-ca\") pod \"f7d9291f-5789-49f7-8596-e3ad311ab0d5\" (UID: \"f7d9291f-5789-49f7-8596-e3ad311ab0d5\") " Sep 10 18:05:34.715867 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:34.715843 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d9291f-5789-49f7-8596-e3ad311ab0d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7d9291f-5789-49f7-8596-e3ad311ab0d5" (UID: "f7d9291f-5789-49f7-8596-e3ad311ab0d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:34.816596 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:34.816565 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d9291f-5789-49f7-8596-e3ad311ab0d5-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:35.457546 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:35.457511 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" Sep 10 18:05:35.457546 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:35.457531 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994" event={"ID":"f7d9291f-5789-49f7-8596-e3ad311ab0d5","Type":"ContainerDied","Data":"6f64e7a55cbd18e2b30c7c5baa00e802b5a0bd060ceffd8577efcb1a3afc6d25"} Sep 10 18:05:35.457755 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:35.457582 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f64e7a55cbd18e2b30c7c5baa00e802b5a0bd060ceffd8577efcb1a3afc6d25" Sep 10 18:05:36.201211 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.201154 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:05:36.228542 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228510 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-envoy\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228701 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228548 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-podinfo\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228701 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-certs\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228701 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228597 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tccn9\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-kube-api-access-tccn9\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228701 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228645 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-credential-socket\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228727 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-token\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228782 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istiod-ca-cert\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228820 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:36.228914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228832 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-data\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.228914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228873 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-socket\") pod \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\" (UID: \"7f9a08da-727d-4c92-9cfd-1caaaf80cef3\") " Sep 10 18:05:36.229174 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.228984 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:36.229174 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229033 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:36.229174 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229150 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:36.229379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229334 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.229379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229348 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.229379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229359 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.229379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229373 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.229603 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.229342 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:36.230841 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.230796 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:36.230841 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.230796 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:05:36.231629 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.231587 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:05:36.232454 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.232430 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-kube-api-access-tccn9" (OuterVolumeSpecName: "kube-api-access-tccn9") pod "7f9a08da-727d-4c92-9cfd-1caaaf80cef3" (UID: "7f9a08da-727d-4c92-9cfd-1caaaf80cef3"). InnerVolumeSpecName "kube-api-access-tccn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:05:36.330280 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.330207 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.330280 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.330275 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.330280 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.330284 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.330505 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.330293 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tccn9\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-kube-api-access-tccn9\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.330505 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.330302 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f9a08da-727d-4c92-9cfd-1caaaf80cef3-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:36.465050 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.464957 2565 generic.go:356] "Generic (PLEG): container finished" podID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerID="1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1" exitCode=0 Sep 10 18:05:36.465050 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.465010 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" Sep 10 18:05:36.465050 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.465036 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" event={"ID":"7f9a08da-727d-4c92-9cfd-1caaaf80cef3","Type":"ContainerDied","Data":"1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1"} Sep 10 18:05:36.465700 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.465089 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh" event={"ID":"7f9a08da-727d-4c92-9cfd-1caaaf80cef3","Type":"ContainerDied","Data":"380df6cfbf7228a1e360324a24781d9d4273df6f06408d42917d77a5f16c4249"} Sep 10 18:05:36.465700 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.465112 2565 scope.go:117] "RemoveContainer" containerID="1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1" Sep 10 18:05:36.475061 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.475043 2565 scope.go:117] "RemoveContainer" containerID="7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5" Sep 10 18:05:36.485492 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.485473 2565 scope.go:117] "RemoveContainer" containerID="1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1" Sep 10 18:05:36.485809 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:05:36.485786 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1\": container with ID starting with 1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1 not found: ID does not exist" containerID="1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1" Sep 10 18:05:36.485928 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.485824 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1"} err="failed to get container status \"1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1\": rpc error: code = NotFound desc = could not find container \"1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1\": container with ID starting with 1626d4538f6adcad24b9b7514fa6c70ba4e72886071d6e760385ba8578509ba1 not found: ID does not exist" Sep 10 18:05:36.485928 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.485845 2565 scope.go:117] "RemoveContainer" containerID="7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5" Sep 10 18:05:36.486163 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:05:36.486131 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5\": container with ID starting with 7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5 not found: ID does not exist" containerID="7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5" Sep 10 18:05:36.486216 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.486165 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5"} err="failed to get container status \"7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5\": rpc error: code = NotFound desc = could not find container \"7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5\": container with ID starting with 7a7150ab0186b1494c0359e20906537b5cc125ae92af832a78c8edaa5370c5c5 not found: ID does not exist" Sep 10 18:05:36.491256 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.491233 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh"] Sep 10 18:05:36.497039 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:36.497016 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b38800-openshift-default-5bb5985786-nptlh"] Sep 10 18:05:37.630216 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:37.630184 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" path="/var/lib/kubelet/pods/7f9a08da-727d-4c92-9cfd-1caaaf80cef3/volumes" Sep 10 18:05:45.534277 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:45.534247 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:46.875775 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:46.875733 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:46.928060 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:46.928023 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr"] Sep 10 18:05:46.928286 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:46.928263 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" podUID="b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" containerName="payload-pre-processing" containerID="cri-o://26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5" gracePeriod=30 Sep 10 18:05:46.957191 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:46.957163 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:47.018740 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.018711 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:47.124417 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.124387 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:47.154160 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.154082 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994"] Sep 10 18:05:47.156320 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.156298 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d439f7-c8994"] Sep 10 18:05:47.182980 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.182961 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:05:47.191188 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.191167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:47.240672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.240636 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-plugins-config-volume\") pod \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " Sep 10 18:05:47.240950 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.240922 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfhhx\" (UniqueName: \"kubernetes.io/projected/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-kube-api-access-cfhhx\") pod \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\" (UID: \"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1\") " Sep 10 18:05:47.242299 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.242257 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" (UID: "b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:47.259698 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.259600 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-kube-api-access-cfhhx" (OuterVolumeSpecName: "kube-api-access-cfhhx") pod "b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" (UID: "b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1"). InnerVolumeSpecName "kube-api-access-cfhhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:05:47.343278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.343221 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:47.343278 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.343274 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfhhx\" (UniqueName: \"kubernetes.io/projected/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1-kube-api-access-cfhhx\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:47.504902 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.504849 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:47.530349 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.530317 2565 generic.go:356] "Generic (PLEG): container finished" podID="b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" containerID="26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5" exitCode=0 Sep 10 18:05:47.530487 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.530356 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" Sep 10 18:05:47.530487 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.530394 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" event={"ID":"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1","Type":"ContainerDied","Data":"26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5"} Sep 10 18:05:47.530487 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.530444 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr" event={"ID":"b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1","Type":"ContainerDied","Data":"08db2de3b2616649fad23519a4dc8b6e9e0904ec0bd81d1c23293da47bf14618"} Sep 10 18:05:47.530487 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.530467 2565 scope.go:117] "RemoveContainer" containerID="26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5" Sep 10 18:05:47.540062 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.540044 2565 scope.go:117] "RemoveContainer" containerID="26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5" Sep 10 18:05:47.540340 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:05:47.540323 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5\": container with ID starting with 26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5 not found: ID does not exist" containerID="26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5" Sep 10 18:05:47.540398 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.540348 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5"} err="failed to get container status \"26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5\": rpc error: code = NotFound desc = could not find container \"26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5\": container with ID starting with 26a63b6e421852a846cff530034ca2a490397b838b878fbd88cb85cac4e609f5 not found: ID does not exist" Sep 10 18:05:47.555309 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.555284 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr"] Sep 10 18:05:47.558556 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.558536 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d439f7-59b7cb45db-vp6mr"] Sep 10 18:05:47.622914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.622881 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" path="/var/lib/kubelet/pods/b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1/volumes" Sep 10 18:05:47.623298 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:47.623283 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7d9291f-5789-49f7-8596-e3ad311ab0d5" path="/var/lib/kubelet/pods/f7d9291f-5789-49f7-8596-e3ad311ab0d5/volumes" Sep 10 18:05:53.903577 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:53.903544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:53.960661 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:53.960630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:54.170353 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.170325 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t"] Sep 10 18:05:54.170578 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.170554 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" containerID="cri-o://c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8" gracePeriod=30 Sep 10 18:05:54.186191 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.186168 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:54.537529 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537456 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8"] Sep 10 18:05:54.537887 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537875 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537889 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537899 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d9291f-5789-49f7-8596-e3ad311ab0d5" containerName="revoke-keys" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537904 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d9291f-5789-49f7-8596-e3ad311ab0d5" containerName="revoke-keys" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537913 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" containerName="payload-pre-processing" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537918 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" containerName="payload-pre-processing" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537926 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" Sep 10 18:05:54.537932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.537931 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" Sep 10 18:05:54.538159 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.538001 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" Sep 10 18:05:54.538159 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.538009 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4641a2e-bcae-4c4e-94ce-71a9e7d9e6c1" containerName="payload-pre-processing" Sep 10 18:05:54.538159 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.538016 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f9a08da-727d-4c92-9cfd-1caaaf80cef3" containerName="istio-proxy" Sep 10 18:05:54.538159 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.538023 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d9291f-5789-49f7-8596-e3ad311ab0d5" containerName="revoke-keys" Sep 10 18:05:54.542378 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.542362 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:54.546248 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.546214 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:05:54.546480 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.546463 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:05:54.609240 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.609198 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8"] Sep 10 18:05:54.612257 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.612207 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4261b-a6c3-4bd6-8517-52be4360af85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8\" (UID: \"5ea4261b-a6c3-4bd6-8517-52be4360af85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:54.659886 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.659861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:54.713546 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.713519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4261b-a6c3-4bd6-8517-52be4360af85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8\" (UID: \"5ea4261b-a6c3-4bd6-8517-52be4360af85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:54.714200 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.714178 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4261b-a6c3-4bd6-8517-52be4360af85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8\" (UID: \"5ea4261b-a6c3-4bd6-8517-52be4360af85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:54.852746 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:54.852667 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:55.135932 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:55.135900 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8"] Sep 10 18:05:55.136975 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:05:55.136939 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea4261b_a6c3_4bd6_8517_52be4360af85.slice/crio-160a0761314df901db0e7a163c21bae441749f65be51242ca07a3017fa0f79a8 WatchSource:0}: Error finding container 160a0761314df901db0e7a163c21bae441749f65be51242ca07a3017fa0f79a8: Status 404 returned error can't find the container with id 160a0761314df901db0e7a163c21bae441749f65be51242ca07a3017fa0f79a8 Sep 10 18:05:55.579002 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:55.578966 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" event={"ID":"5ea4261b-a6c3-4bd6-8517-52be4360af85","Type":"ContainerStarted","Data":"6b97801eed81dc232d2c9729255255aaf3b634a4a109c1f9ab9465bc3c8effd7"} Sep 10 18:05:55.579159 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:55.579027 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" event={"ID":"5ea4261b-a6c3-4bd6-8517-52be4360af85","Type":"ContainerStarted","Data":"160a0761314df901db0e7a163c21bae441749f65be51242ca07a3017fa0f79a8"} Sep 10 18:05:55.601188 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:55.601142 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" podStartSLOduration=1.601128769 podStartE2EDuration="1.601128769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:05:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:05:55.597760053 +0000 UTC m=+1484.479485196" watchObservedRunningTime="2026-09-10 18:05:55.601128769 +0000 UTC m=+1484.482853938" Sep 10 18:05:56.585837 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:56.585800 2565 generic.go:356] "Generic (PLEG): container finished" podID="5ea4261b-a6c3-4bd6-8517-52be4360af85" containerID="6b97801eed81dc232d2c9729255255aaf3b634a4a109c1f9ab9465bc3c8effd7" exitCode=0 Sep 10 18:05:56.585837 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:56.585838 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" event={"ID":"5ea4261b-a6c3-4bd6-8517-52be4360af85","Type":"ContainerDied","Data":"6b97801eed81dc232d2c9729255255aaf3b634a4a109c1f9ab9465bc3c8effd7"} Sep 10 18:05:57.724620 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:57.724594 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:57.845429 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:57.845357 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4261b-a6c3-4bd6-8517-52be4360af85-maas-api-service-ca\") pod \"5ea4261b-a6c3-4bd6-8517-52be4360af85\" (UID: \"5ea4261b-a6c3-4bd6-8517-52be4360af85\") " Sep 10 18:05:57.845686 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:57.845663 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea4261b-a6c3-4bd6-8517-52be4360af85-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ea4261b-a6c3-4bd6-8517-52be4360af85" (UID: "5ea4261b-a6c3-4bd6-8517-52be4360af85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:57.946548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:57.946521 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4261b-a6c3-4bd6-8517-52be4360af85-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:58.598322 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:58.598296 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" Sep 10 18:05:58.598471 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:58.598346 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8" event={"ID":"5ea4261b-a6c3-4bd6-8517-52be4360af85","Type":"ContainerDied","Data":"160a0761314df901db0e7a163c21bae441749f65be51242ca07a3017fa0f79a8"} Sep 10 18:05:58.598471 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:58.598395 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="160a0761314df901db0e7a163c21bae441749f65be51242ca07a3017fa0f79a8" Sep 10 18:05:59.422758 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.422736 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:05:59.562045 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.561958 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-certs\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562045 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.561990 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-socket\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562045 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562023 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-token\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562045 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562038 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-envoy\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562083 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-data\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562119 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-credential-socket\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562178 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e43be7e3-3df7-4696-925f-be3ea041f9a7-istiod-ca-cert\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562217 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562245 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-podinfo\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562279 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:59.562369 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562307 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7m54\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-kube-api-access-h7m54\") pod \"e43be7e3-3df7-4696-925f-be3ea041f9a7\" (UID: \"e43be7e3-3df7-4696-925f-be3ea041f9a7\") " Sep 10 18:05:59.562709 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562609 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:59.562764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562718 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.562764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562738 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.562764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562751 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.562976 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.562819 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:59.563213 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.563183 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e43be7e3-3df7-4696-925f-be3ea041f9a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:05:59.563343 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.563243 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:05:59.564629 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.564607 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:05:59.564894 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.564864 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:05:59.564999 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.564978 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-kube-api-access-h7m54" (OuterVolumeSpecName: "kube-api-access-h7m54") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "kube-api-access-h7m54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:05:59.565247 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.565202 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e43be7e3-3df7-4696-925f-be3ea041f9a7" (UID: "e43be7e3-3df7-4696-925f-be3ea041f9a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:05:59.608906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.608879 2565 generic.go:356] "Generic (PLEG): container finished" podID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerID="c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8" exitCode=0 Sep 10 18:05:59.609017 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.608925 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" Sep 10 18:05:59.609017 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.608928 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" event={"ID":"e43be7e3-3df7-4696-925f-be3ea041f9a7","Type":"ContainerDied","Data":"c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8"} Sep 10 18:05:59.609143 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.609061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t" event={"ID":"e43be7e3-3df7-4696-925f-be3ea041f9a7","Type":"ContainerDied","Data":"eb12276652f28ccdca8591af26b28f817441f358e6bb8fc7e04def6b3f8a8f9b"} Sep 10 18:05:59.609143 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.609104 2565 scope.go:117] "RemoveContainer" containerID="c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8" Sep 10 18:05:59.618738 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.618718 2565 scope.go:117] "RemoveContainer" containerID="b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11" Sep 10 18:05:59.627800 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.627784 2565 scope.go:117] "RemoveContainer" containerID="c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8" Sep 10 18:05:59.628063 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:05:59.628047 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8\": container with ID starting with c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8 not found: ID does not exist" containerID="c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8" Sep 10 18:05:59.628115 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.628070 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8"} err="failed to get container status \"c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8\": rpc error: code = NotFound desc = could not find container \"c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8\": container with ID starting with c1aa29558df7391ac69b7767fe2fbd0d07903a50ec249db674873c087938fcb8 not found: ID does not exist" Sep 10 18:05:59.628115 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.628086 2565 scope.go:117] "RemoveContainer" containerID="b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11" Sep 10 18:05:59.628376 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:05:59.628359 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11\": container with ID starting with b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11 not found: ID does not exist" containerID="b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11" Sep 10 18:05:59.628421 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.628383 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11"} err="failed to get container status \"b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11\": rpc error: code = NotFound desc = could not find container \"b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11\": container with ID starting with b07b45f4eb186fe37633a53794ad4cb13697753c61e10a456872ea8ae6437d11 not found: ID does not exist" Sep 10 18:05:59.638125 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.638103 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t"] Sep 10 18:05:59.640241 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.640209 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d439f7-openshift-default-65f7b78c8d-6gn4t"] Sep 10 18:05:59.664301 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.664280 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.664301 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.664299 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.664417 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.664307 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.664417 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.664316 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e43be7e3-3df7-4696-925f-be3ea041f9a7-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.664417 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.664332 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e43be7e3-3df7-4696-925f-be3ea041f9a7-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:05:59.664417 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:05:59.664340 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7m54\" (UniqueName: \"kubernetes.io/projected/e43be7e3-3df7-4696-925f-be3ea041f9a7-kube-api-access-h7m54\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:01.628546 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:01.623747 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" path="/var/lib/kubelet/pods/e43be7e3-3df7-4696-925f-be3ea041f9a7/volumes" Sep 10 18:06:05.282370 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.282332 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls"] Sep 10 18:06:05.283385 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.283331 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" podUID="ccaf994e-3c89-4158-a517-d68235721318" containerName="payload-pre-processing" containerID="cri-o://00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc" gracePeriod=30 Sep 10 18:06:05.514344 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.513752 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8"] Sep 10 18:06:05.518987 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.518956 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-971a94-sx5v8"] Sep 10 18:06:05.568719 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.568698 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 18:06:05.622995 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.622963 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea4261b-a6c3-4bd6-8517-52be4360af85" path="/var/lib/kubelet/pods/5ea4261b-a6c3-4bd6-8517-52be4360af85/volumes" Sep 10 18:06:05.644060 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.644031 2565 generic.go:356] "Generic (PLEG): container finished" podID="ccaf994e-3c89-4158-a517-d68235721318" containerID="00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc" exitCode=0 Sep 10 18:06:05.644209 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.644056 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" event={"ID":"ccaf994e-3c89-4158-a517-d68235721318","Type":"ContainerDied","Data":"00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc"} Sep 10 18:06:05.644209 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.644073 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" Sep 10 18:06:05.644209 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.644104 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls" event={"ID":"ccaf994e-3c89-4158-a517-d68235721318","Type":"ContainerDied","Data":"43a59ceb5a020dadd283867ffa0076431e8c216ab392f3310fc55ba5e74ec857"} Sep 10 18:06:05.644209 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.644126 2565 scope.go:117] "RemoveContainer" containerID="00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc" Sep 10 18:06:05.652759 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.652743 2565 scope.go:117] "RemoveContainer" containerID="00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc" Sep 10 18:06:05.653040 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:06:05.653022 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc\": container with ID starting with 00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc not found: ID does not exist" containerID="00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc" Sep 10 18:06:05.653085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.653049 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc"} err="failed to get container status \"00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc\": rpc error: code = NotFound desc = could not find container \"00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc\": container with ID starting with 00d073aaf624964e9843d914b3665b1656e4436770483c8e752a9abadbf2c6fc not found: ID does not exist" Sep 10 18:06:05.719556 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.719534 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccaf994e-3c89-4158-a517-d68235721318-plugins-config-volume\") pod \"ccaf994e-3c89-4158-a517-d68235721318\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " Sep 10 18:06:05.719675 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.719606 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5npj4\" (UniqueName: \"kubernetes.io/projected/ccaf994e-3c89-4158-a517-d68235721318-kube-api-access-5npj4\") pod \"ccaf994e-3c89-4158-a517-d68235721318\" (UID: \"ccaf994e-3c89-4158-a517-d68235721318\") " Sep 10 18:06:05.719911 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.719891 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccaf994e-3c89-4158-a517-d68235721318-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ccaf994e-3c89-4158-a517-d68235721318" (UID: "ccaf994e-3c89-4158-a517-d68235721318"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:06:05.721537 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.721518 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccaf994e-3c89-4158-a517-d68235721318-kube-api-access-5npj4" (OuterVolumeSpecName: "kube-api-access-5npj4") pod "ccaf994e-3c89-4158-a517-d68235721318" (UID: "ccaf994e-3c89-4158-a517-d68235721318"). InnerVolumeSpecName "kube-api-access-5npj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:06:05.821019 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.820966 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccaf994e-3c89-4158-a517-d68235721318-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:05.821019 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.820986 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5npj4\" (UniqueName: \"kubernetes.io/projected/ccaf994e-3c89-4158-a517-d68235721318-kube-api-access-5npj4\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:05.969343 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.969312 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls"] Sep 10 18:06:05.974501 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:05.974475 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-971a94-7645b49966-twvls"] Sep 10 18:06:06.025579 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:06.025549 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:06.126336 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:06.126278 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:07.623623 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:07.623588 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccaf994e-3c89-4158-a517-d68235721318" path="/var/lib/kubelet/pods/ccaf994e-3c89-4158-a517-d68235721318/volumes" Sep 10 18:06:08.815479 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:08.815443 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:11.958050 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:11.958007 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:12.057429 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.057358 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:12.205045 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.205016 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj"] Sep 10 18:06:12.205288 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.205265 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" containerID="cri-o://d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba" gracePeriod=30 Sep 10 18:06:12.221445 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.221419 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:12.548385 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548345 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv"] Sep 10 18:06:12.548803 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548786 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" Sep 10 18:06:12.548803 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548803 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548819 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccaf994e-3c89-4158-a517-d68235721318" containerName="payload-pre-processing" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548825 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccaf994e-3c89-4158-a517-d68235721318" containerName="payload-pre-processing" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548836 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548844 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548852 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4261b-a6c3-4bd6-8517-52be4360af85" containerName="revoke-keys" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548858 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4261b-a6c3-4bd6-8517-52be4360af85" containerName="revoke-keys" Sep 10 18:06:12.548931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548926 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccaf994e-3c89-4158-a517-d68235721318" containerName="payload-pre-processing" Sep 10 18:06:12.549180 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548936 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" Sep 10 18:06:12.549180 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.548946 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4261b-a6c3-4bd6-8517-52be4360af85" containerName="revoke-keys" Sep 10 18:06:12.551914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.551900 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:12.557517 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.557497 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:06:12.557649 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.557632 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:06:12.563894 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.563872 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv"] Sep 10 18:06:12.685884 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.685849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08002618-d323-475e-ad61-5658397b4ed6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv\" (UID: \"08002618-d323-475e-ad61-5658397b4ed6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:12.740843 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.740818 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:12.786854 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.786830 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08002618-d323-475e-ad61-5658397b4ed6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv\" (UID: \"08002618-d323-475e-ad61-5658397b4ed6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:12.787404 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.787387 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08002618-d323-475e-ad61-5658397b4ed6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv\" (UID: \"08002618-d323-475e-ad61-5658397b4ed6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:12.862900 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:12.862816 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:13.135117 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:13.135091 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv"] Sep 10 18:06:13.136556 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:06:13.136526 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08002618_d323_475e_ad61_5658397b4ed6.slice/crio-04f921cb0ce03ed4eaad235953aae7d7182fbe66f3d9ed1dce8b794a045436e4 WatchSource:0}: Error finding container 04f921cb0ce03ed4eaad235953aae7d7182fbe66f3d9ed1dce8b794a045436e4: Status 404 returned error can't find the container with id 04f921cb0ce03ed4eaad235953aae7d7182fbe66f3d9ed1dce8b794a045436e4 Sep 10 18:06:13.694180 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:13.694149 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" event={"ID":"08002618-d323-475e-ad61-5658397b4ed6","Type":"ContainerStarted","Data":"ef77e001f5518ef8b7ffdec034bf8e25dbc5d6cfbbcbe255cf5b2de46f6df749"} Sep 10 18:06:13.694371 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:13.694196 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" event={"ID":"08002618-d323-475e-ad61-5658397b4ed6","Type":"ContainerStarted","Data":"04f921cb0ce03ed4eaad235953aae7d7182fbe66f3d9ed1dce8b794a045436e4"} Sep 10 18:06:13.720986 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:13.720942 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" podStartSLOduration=1.720928726 podStartE2EDuration="1.720928726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:06:13.719339527 +0000 UTC m=+1502.601064665" watchObservedRunningTime="2026-09-10 18:06:13.720928726 +0000 UTC m=+1502.602653865" Sep 10 18:06:14.701920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:14.701888 2565 generic.go:356] "Generic (PLEG): container finished" podID="08002618-d323-475e-ad61-5658397b4ed6" containerID="ef77e001f5518ef8b7ffdec034bf8e25dbc5d6cfbbcbe255cf5b2de46f6df749" exitCode=0 Sep 10 18:06:14.702331 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:14.701974 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" event={"ID":"08002618-d323-475e-ad61-5658397b4ed6","Type":"ContainerDied","Data":"ef77e001f5518ef8b7ffdec034bf8e25dbc5d6cfbbcbe255cf5b2de46f6df749"} Sep 10 18:06:15.839366 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:15.839343 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:15.916356 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:15.916332 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08002618-d323-475e-ad61-5658397b4ed6-maas-api-service-ca\") pod \"08002618-d323-475e-ad61-5658397b4ed6\" (UID: \"08002618-d323-475e-ad61-5658397b4ed6\") " Sep 10 18:06:15.916680 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:15.916660 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08002618-d323-475e-ad61-5658397b4ed6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08002618-d323-475e-ad61-5658397b4ed6" (UID: "08002618-d323-475e-ad61-5658397b4ed6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:06:16.016959 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:16.016899 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08002618-d323-475e-ad61-5658397b4ed6-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:16.432924 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:16.432899 2565 patch_prober.go:28] interesting pod/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 18:06:16.433049 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:16.432948 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 18:06:16.714130 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:16.714060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" event={"ID":"08002618-d323-475e-ad61-5658397b4ed6","Type":"ContainerDied","Data":"04f921cb0ce03ed4eaad235953aae7d7182fbe66f3d9ed1dce8b794a045436e4"} Sep 10 18:06:16.714130 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:16.714110 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04f921cb0ce03ed4eaad235953aae7d7182fbe66f3d9ed1dce8b794a045436e4" Sep 10 18:06:16.714130 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:16.714070 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv" Sep 10 18:06:17.466941 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.466882 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 18:06:17.527185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527159 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-certs\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527381 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527362 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-credential-socket\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527437 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527417 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-socket\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527482 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527435 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:17.527521 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527488 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istiod-ca-cert\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527521 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527509 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-token\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527608 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527542 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-envoy\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527608 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527573 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-data\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527608 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527598 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-podinfo\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527624 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88g82\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-kube-api-access-88g82\") pod \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\" (UID: \"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af\") " Sep 10 18:06:17.527764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527701 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:17.527764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527731 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:17.527931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.527870 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:06:17.528207 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.528062 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.528207 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.528083 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.528207 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.528096 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.528428 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.528240 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.528428 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.528182 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-data" (OuterVolumeSpecName: "istio-data") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:17.530298 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.530264 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:06:17.530400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.530309 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:17.530400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.530352 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-kube-api-access-88g82" (OuterVolumeSpecName: "kube-api-access-88g82") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "kube-api-access-88g82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:06:17.530400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.530368 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-token" (OuterVolumeSpecName: "istio-token") pod "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" (UID: "26a54ca7-2ec3-4bfd-9bf4-a546e34a13af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:06:17.629571 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.629402 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.629571 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.629424 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.629571 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.629433 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.629571 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.629441 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.629571 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.629449 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88g82\" (UniqueName: \"kubernetes.io/projected/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af-kube-api-access-88g82\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:17.721598 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.721542 2565 generic.go:356] "Generic (PLEG): container finished" podID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerID="d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba" exitCode=0 Sep 10 18:06:17.721598 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.721572 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" event={"ID":"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af","Type":"ContainerDied","Data":"d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba"} Sep 10 18:06:17.721598 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.721581 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" Sep 10 18:06:17.721598 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.721603 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj" event={"ID":"26a54ca7-2ec3-4bfd-9bf4-a546e34a13af","Type":"ContainerDied","Data":"6dda6211ff82d9ef28d47c37a230655244f0f6963ed58d3373297164f9a09f4c"} Sep 10 18:06:17.721830 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.721623 2565 scope.go:117] "RemoveContainer" containerID="d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba" Sep 10 18:06:17.733994 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.733959 2565 scope.go:117] "RemoveContainer" containerID="4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606" Sep 10 18:06:17.741292 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.741267 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj"] Sep 10 18:06:17.743462 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.743440 2565 scope.go:117] "RemoveContainer" containerID="d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba" Sep 10 18:06:17.743762 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:06:17.743741 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba\": container with ID starting with d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba not found: ID does not exist" containerID="d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba" Sep 10 18:06:17.743847 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.743773 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba"} err="failed to get container status \"d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba\": rpc error: code = NotFound desc = could not find container \"d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba\": container with ID starting with d8e8d8db84464b9ed85a20fa445baad465b711aa9099b091442c1808df43b2ba not found: ID does not exist" Sep 10 18:06:17.743847 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.743797 2565 scope.go:117] "RemoveContainer" containerID="4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606" Sep 10 18:06:17.744084 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:06:17.744066 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606\": container with ID starting with 4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606 not found: ID does not exist" containerID="4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606" Sep 10 18:06:17.744147 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.744092 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606"} err="failed to get container status \"4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606\": rpc error: code = NotFound desc = could not find container \"4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606\": container with ID starting with 4932767451e1b63b58f99241ecc6cc771cf80a0037b93672cd455a272eaf9606 not found: ID does not exist" Sep 10 18:06:17.745823 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:17.745804 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-971a94-openshift-default-6dc5945f88-tt2gj"] Sep 10 18:06:19.622511 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:19.622476 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" path="/var/lib/kubelet/pods/26a54ca7-2ec3-4bfd-9bf4-a546e34a13af/volumes" Sep 10 18:06:23.235404 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.235367 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:23.290455 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.290427 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p"] Sep 10 18:06:23.290637 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.290613 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" podUID="1bd1a003-4b12-4142-b660-6cc825685467" containerName="payload-pre-processing" containerID="cri-o://71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66" gracePeriod=30 Sep 10 18:06:23.322863 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.322835 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:23.387509 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.387481 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:23.497375 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.497248 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:23.538236 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.538195 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv"] Sep 10 18:06:23.540452 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.540424 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-09fbb5-nz8xv"] Sep 10 18:06:23.553356 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.553336 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:06:23.557072 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.557048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:23.578120 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.578085 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6775\" (UniqueName: \"kubernetes.io/projected/1bd1a003-4b12-4142-b660-6cc825685467-kube-api-access-j6775\") pod \"1bd1a003-4b12-4142-b660-6cc825685467\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " Sep 10 18:06:23.578242 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.578188 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd1a003-4b12-4142-b660-6cc825685467-plugins-config-volume\") pod \"1bd1a003-4b12-4142-b660-6cc825685467\" (UID: \"1bd1a003-4b12-4142-b660-6cc825685467\") " Sep 10 18:06:23.578583 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.578547 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bd1a003-4b12-4142-b660-6cc825685467-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1bd1a003-4b12-4142-b660-6cc825685467" (UID: "1bd1a003-4b12-4142-b660-6cc825685467"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:06:23.581271 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.581204 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bd1a003-4b12-4142-b660-6cc825685467-kube-api-access-j6775" (OuterVolumeSpecName: "kube-api-access-j6775") pod "1bd1a003-4b12-4142-b660-6cc825685467" (UID: "1bd1a003-4b12-4142-b660-6cc825685467"). InnerVolumeSpecName "kube-api-access-j6775". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:06:23.624056 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.624004 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08002618-d323-475e-ad61-5658397b4ed6" path="/var/lib/kubelet/pods/08002618-d323-475e-ad61-5658397b4ed6/volumes" Sep 10 18:06:23.679214 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.679186 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6775\" (UniqueName: \"kubernetes.io/projected/1bd1a003-4b12-4142-b660-6cc825685467-kube-api-access-j6775\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:23.679214 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.679209 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd1a003-4b12-4142-b660-6cc825685467-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:23.756488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.756429 2565 generic.go:356] "Generic (PLEG): container finished" podID="1bd1a003-4b12-4142-b660-6cc825685467" containerID="71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66" exitCode=0 Sep 10 18:06:23.756488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.756475 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" Sep 10 18:06:23.756639 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.756472 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" event={"ID":"1bd1a003-4b12-4142-b660-6cc825685467","Type":"ContainerDied","Data":"71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66"} Sep 10 18:06:23.756639 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.756615 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p" event={"ID":"1bd1a003-4b12-4142-b660-6cc825685467","Type":"ContainerDied","Data":"41f3257306dbc2bcd435079030ee6e7e72811069825b9baa9906b84caeaa0cad"} Sep 10 18:06:23.756720 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.756649 2565 scope.go:117] "RemoveContainer" containerID="71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66" Sep 10 18:06:23.765622 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.765604 2565 scope.go:117] "RemoveContainer" containerID="71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66" Sep 10 18:06:23.765881 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:06:23.765862 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66\": container with ID starting with 71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66 not found: ID does not exist" containerID="71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66" Sep 10 18:06:23.765942 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.765886 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66"} err="failed to get container status \"71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66\": rpc error: code = NotFound desc = could not find container \"71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66\": container with ID starting with 71e73b172a4d87eafc7711fd356c221eb368a585f6c408a3ffef0a666a7aff66 not found: ID does not exist" Sep 10 18:06:23.776660 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.776637 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p"] Sep 10 18:06:23.782421 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.782401 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-09fbb5-754d84b678-2sf9p"] Sep 10 18:06:23.889746 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:23.889723 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:25.622483 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:25.622451 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bd1a003-4b12-4142-b660-6cc825685467" path="/var/lib/kubelet/pods/1bd1a003-4b12-4142-b660-6cc825685467/volumes" Sep 10 18:06:30.138978 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:30.138943 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:30.240770 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:30.240731 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:30.399819 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:30.399732 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl"] Sep 10 18:06:30.400034 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:30.399990 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" containerID="cri-o://63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d" gracePeriod=30 Sep 10 18:06:30.418129 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:30.418087 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:32.190822 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:32.190788 2565 patch_prober.go:28] interesting pod/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 18:06:32.191252 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:32.190846 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 18:06:35.654813 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.654784 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:06:35.792540 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792502 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab242f-9d81-409d-abda-7577f121451c-istiod-ca-cert\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.792733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792554 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-certs\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.792733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792587 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-envoy\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.792733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792613 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-data\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.792733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792663 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdjr5\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-kube-api-access-cdjr5\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.792733 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792691 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-socket\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.793005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792855 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:35.793005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792897 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:35.793005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792915 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dab242f-9d81-409d-abda-7577f121451c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:06:35.793005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792949 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:35.793005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.792956 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-istio-token\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.793363 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793041 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab242f-9d81-409d-abda-7577f121451c-istio-podinfo\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.793363 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793101 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-credential-socket\") pod \"3dab242f-9d81-409d-abda-7577f121451c\" (UID: \"3dab242f-9d81-409d-abda-7577f121451c\") " Sep 10 18:06:35.793489 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793383 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:35.793548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793491 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.793548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793511 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab242f-9d81-409d-abda-7577f121451c-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.793548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793526 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.793548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793537 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.793764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.793549 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.795072 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.795045 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:06:35.795072 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.795062 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3dab242f-9d81-409d-abda-7577f121451c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:06:35.795347 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.795316 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-kube-api-access-cdjr5" (OuterVolumeSpecName: "kube-api-access-cdjr5") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "kube-api-access-cdjr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:06:35.795476 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.795363 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3dab242f-9d81-409d-abda-7577f121451c" (UID: "3dab242f-9d81-409d-abda-7577f121451c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:06:35.821005 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.820984 2565 generic.go:356] "Generic (PLEG): container finished" podID="3dab242f-9d81-409d-abda-7577f121451c" containerID="63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d" exitCode=0 Sep 10 18:06:35.821089 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.821029 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" Sep 10 18:06:35.821089 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.821034 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" event={"ID":"3dab242f-9d81-409d-abda-7577f121451c","Type":"ContainerDied","Data":"63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d"} Sep 10 18:06:35.821089 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.821073 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl" event={"ID":"3dab242f-9d81-409d-abda-7577f121451c","Type":"ContainerDied","Data":"4fe553393b8e89a26f5821a82ec1580a2e454b868311e71a3e39d3bec22e2d79"} Sep 10 18:06:35.821195 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.821095 2565 scope.go:117] "RemoveContainer" containerID="63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d" Sep 10 18:06:35.830030 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.830015 2565 scope.go:117] "RemoveContainer" containerID="dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb" Sep 10 18:06:35.837666 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.837650 2565 scope.go:117] "RemoveContainer" containerID="63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d" Sep 10 18:06:35.837947 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:06:35.837929 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d\": container with ID starting with 63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d not found: ID does not exist" containerID="63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d" Sep 10 18:06:35.837997 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.837956 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d"} err="failed to get container status \"63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d\": rpc error: code = NotFound desc = could not find container \"63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d\": container with ID starting with 63d70d683c5a1c1ad4bf9ce3f0125b03b2439c8a6c6a98c3bc53b173189b501d not found: ID does not exist" Sep 10 18:06:35.837997 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.837973 2565 scope.go:117] "RemoveContainer" containerID="dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb" Sep 10 18:06:35.838215 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:06:35.838198 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb\": container with ID starting with dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb not found: ID does not exist" containerID="dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb" Sep 10 18:06:35.838295 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.838241 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb"} err="failed to get container status \"dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb\": rpc error: code = NotFound desc = could not find container \"dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb\": container with ID starting with dd9a2c2d50400b24444a7a747ea8cc379c881d3e7a678baba503b4060bb559fb not found: ID does not exist" Sep 10 18:06:35.843314 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.843293 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl"] Sep 10 18:06:35.849985 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.849963 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-09fbb5-openshift-default-76bc756d7b-bfwrl"] Sep 10 18:06:35.894296 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.894265 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab242f-9d81-409d-abda-7577f121451c-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.894296 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.894295 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab242f-9d81-409d-abda-7577f121451c-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.894296 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.894304 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjr5\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-kube-api-access-cdjr5\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:35.894445 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:35.894313 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab242f-9d81-409d-abda-7577f121451c-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:06:37.622430 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:37.622394 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dab242f-9d81-409d-abda-7577f121451c" path="/var/lib/kubelet/pods/3dab242f-9d81-409d-abda-7577f121451c/volumes" Sep 10 18:06:39.713298 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:39.713263 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:39.858474 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:39.858433 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:40.933325 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:40.933293 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:41.354385 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.354306 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:41.642641 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.642558 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq"] Sep 10 18:06:41.642967 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.642949 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" Sep 10 18:06:41.642967 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.642966 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.642985 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bd1a003-4b12-4142-b660-6cc825685467" containerName="payload-pre-processing" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.642991 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bd1a003-4b12-4142-b660-6cc825685467" containerName="payload-pre-processing" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643001 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643007 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643017 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08002618-d323-475e-ad61-5658397b4ed6" containerName="revoke-keys" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643022 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="08002618-d323-475e-ad61-5658397b4ed6" containerName="revoke-keys" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643032 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643037 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643044 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" Sep 10 18:06:41.643095 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643049 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643108 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e43be7e3-3df7-4696-925f-be3ea041f9a7" containerName="istio-proxy" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643118 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bd1a003-4b12-4142-b660-6cc825685467" containerName="payload-pre-processing" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643124 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643130 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dab242f-9d81-409d-abda-7577f121451c" containerName="istio-proxy" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643137 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="08002618-d323-475e-ad61-5658397b4ed6" containerName="revoke-keys" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643146 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" Sep 10 18:06:41.643465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.643152 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a54ca7-2ec3-4bfd-9bf4-a546e34a13af" containerName="istio-proxy" Sep 10 18:06:41.647539 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.647508 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.650465 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.650445 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-732455-openshift-default-dockercfg-q8cv2\"" Sep 10 18:06:41.661059 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.661034 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq"] Sep 10 18:06:41.665419 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.665381 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:41.750934 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.750871 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istiod-ca-cert\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751123 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.750993 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-envoy\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751123 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-token\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751123 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-credential-socket\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751314 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751124 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-podinfo\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751314 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751184 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-socket\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751314 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-certs\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751314 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751279 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-data\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.751531 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.751333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tl92\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-kube-api-access-6tl92\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852489 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-data\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tl92\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-kube-api-access-6tl92\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istiod-ca-cert\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-envoy\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852645 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-token\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852882 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-credential-socket\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852882 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-podinfo\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.852882 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852878 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-data\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.853039 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852958 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-socket\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.853039 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.852992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-certs\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.853039 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.853004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-credential-socket\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.853401 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.853377 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istiod-ca-cert\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.853506 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.853474 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-socket\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.853583 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.853565 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-certs\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.855379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.855356 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-envoy\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.855603 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.855582 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-podinfo\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.860847 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.860822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-token\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.861053 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.861031 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tl92\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-kube-api-access-6tl92\") pod \"e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:41.871564 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.871544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:41.957372 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:41.957344 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:42.233630 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.233602 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq"] Sep 10 18:06:42.235111 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:06:42.235081 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0d68cb_24fd_47d8_9e6d_e9c19c5974a0.slice/crio-498c9dbb4fb3c99664ddf5c526a8367a7ca7778b86bb38bb9f8c128be522dab9 WatchSource:0}: Error finding container 498c9dbb4fb3c99664ddf5c526a8367a7ca7778b86bb38bb9f8c128be522dab9: Status 404 returned error can't find the container with id 498c9dbb4fb3c99664ddf5c526a8367a7ca7778b86bb38bb9f8c128be522dab9 Sep 10 18:06:42.237440 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.237393 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:06:42.237527 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.237487 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:06:42.237563 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.237533 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:06:42.533696 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.533602 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:42.634919 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.634889 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:42.866188 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.866101 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" event={"ID":"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0","Type":"ContainerStarted","Data":"e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10"} Sep 10 18:06:42.866188 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.866157 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" event={"ID":"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0","Type":"ContainerStarted","Data":"498c9dbb4fb3c99664ddf5c526a8367a7ca7778b86bb38bb9f8c128be522dab9"} Sep 10 18:06:42.889747 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.889685 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podStartSLOduration=1.889666345 podStartE2EDuration="1.889666345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:06:42.889537961 +0000 UTC m=+1531.771263103" watchObservedRunningTime="2026-09-10 18:06:42.889666345 +0000 UTC m=+1531.771391685" Sep 10 18:06:42.958509 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:42.958488 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:43.963205 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:43.963172 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:44.764895 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:44.764851 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:44.862149 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:44.862112 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:44.877991 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:44.877965 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:44.878918 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:44.878895 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:06:46.033115 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.033084 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:46.093626 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.093593 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:46.194655 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.194617 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:46.495401 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.495364 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:46.593668 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.593635 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:06:46.594770 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.594739 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt"] Sep 10 18:06:46.599257 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.599236 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.603357 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.603327 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-732455\"" Sep 10 18:06:46.608396 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.608370 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt"] Sep 10 18:06:46.622127 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.622099 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m"] Sep 10 18:06:46.625653 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.625631 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.628427 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.628410 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-732455-dockercfg-jxqj6\"" Sep 10 18:06:46.637529 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.637503 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m"] Sep 10 18:06:46.705019 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.704981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jhmc\" (UniqueName: \"kubernetes.io/projected/1da3a56f-9e34-4794-86b9-ebd064e75b0e-kube-api-access-4jhmc\") pod \"payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.705213 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.705053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da3a56f-9e34-4794-86b9-ebd064e75b0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.705213 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.705088 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06af970f-b41a-43aa-96fe-edba191c52a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.705213 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.705157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sltb9\" (UniqueName: \"kubernetes.io/projected/06af970f-b41a-43aa-96fe-edba191c52a7-kube-api-access-sltb9\") pod \"payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.806587 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.806496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jhmc\" (UniqueName: \"kubernetes.io/projected/1da3a56f-9e34-4794-86b9-ebd064e75b0e-kube-api-access-4jhmc\") pod \"payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.806587 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.806564 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da3a56f-9e34-4794-86b9-ebd064e75b0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.806931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.806612 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06af970f-b41a-43aa-96fe-edba191c52a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.806931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.806695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sltb9\" (UniqueName: \"kubernetes.io/projected/06af970f-b41a-43aa-96fe-edba191c52a7-kube-api-access-sltb9\") pod \"payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.807316 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.807294 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da3a56f-9e34-4794-86b9-ebd064e75b0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.807365 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.807319 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06af970f-b41a-43aa-96fe-edba191c52a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.814906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.814882 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sltb9\" (UniqueName: \"kubernetes.io/projected/06af970f-b41a-43aa-96fe-edba191c52a7-kube-api-access-sltb9\") pod \"payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.815008 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.814937 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jhmc\" (UniqueName: \"kubernetes.io/projected/1da3a56f-9e34-4794-86b9-ebd064e75b0e-kube-api-access-4jhmc\") pod \"payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:46.914420 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.914392 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:46.938436 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:46.938407 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:47.192715 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.192679 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt"] Sep 10 18:06:47.194522 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:06:47.194495 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06af970f_b41a_43aa_96fe_edba191c52a7.slice/crio-4b8817677c181bdd78a1f0f877aa3a50c9387a391fb6648018a735ee12abf809 WatchSource:0}: Error finding container 4b8817677c181bdd78a1f0f877aa3a50c9387a391fb6648018a735ee12abf809: Status 404 returned error can't find the container with id 4b8817677c181bdd78a1f0f877aa3a50c9387a391fb6648018a735ee12abf809 Sep 10 18:06:47.221985 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.221958 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m"] Sep 10 18:06:47.233354 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:06:47.233331 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da3a56f_9e34_4794_86b9_ebd064e75b0e.slice/crio-a564524d735880c190f7d4df9b4ec406e8d4d84bc50a129e1e01985b34381096 WatchSource:0}: Error finding container a564524d735880c190f7d4df9b4ec406e8d4d84bc50a129e1e01985b34381096: Status 404 returned error can't find the container with id a564524d735880c190f7d4df9b4ec406e8d4d84bc50a129e1e01985b34381096 Sep 10 18:06:47.896890 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.896851 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" event={"ID":"1da3a56f-9e34-4794-86b9-ebd064e75b0e","Type":"ContainerStarted","Data":"62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd"} Sep 10 18:06:47.897054 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.896922 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:47.897054 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.896947 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" event={"ID":"1da3a56f-9e34-4794-86b9-ebd064e75b0e","Type":"ContainerStarted","Data":"a564524d735880c190f7d4df9b4ec406e8d4d84bc50a129e1e01985b34381096"} Sep 10 18:06:47.898511 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.898487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" event={"ID":"06af970f-b41a-43aa-96fe-edba191c52a7","Type":"ContainerStarted","Data":"f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f"} Sep 10 18:06:47.898626 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.898530 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" event={"ID":"06af970f-b41a-43aa-96fe-edba191c52a7","Type":"ContainerStarted","Data":"4b8817677c181bdd78a1f0f877aa3a50c9387a391fb6648018a735ee12abf809"} Sep 10 18:06:47.898683 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.898628 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:06:47.917621 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.917577 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" podStartSLOduration=1.917562754 podStartE2EDuration="1.917562754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:06:47.915875766 +0000 UTC m=+1536.797600910" watchObservedRunningTime="2026-09-10 18:06:47.917562754 +0000 UTC m=+1536.799287895" Sep 10 18:06:47.933181 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:47.933143 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" podStartSLOduration=1.9331309079999999 podStartE2EDuration="1.933130908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:06:47.931905643 +0000 UTC m=+1536.813630786" watchObservedRunningTime="2026-09-10 18:06:47.933130908 +0000 UTC m=+1536.814856087" Sep 10 18:06:58.906657 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:58.906607 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:06:58.907130 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:06:58.906816 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:07:00.905144 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.904594 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 18:07:00.905144 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.904675 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 18:07:00.971897 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.971869 2565 generic.go:356] "Generic (PLEG): container finished" podID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerID="e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10" exitCode=0 Sep 10 18:07:00.972072 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.971946 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" event={"ID":"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0","Type":"ContainerDied","Data":"e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10"} Sep 10 18:07:00.972443 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.972425 2565 scope.go:117] "RemoveContainer" containerID="e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10" Sep 10 18:07:00.973530 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.973502 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:07:00.973585 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.973570 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:07:00.973621 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:00.973610 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:07:01.983648 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:01.983602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" event={"ID":"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0","Type":"ContainerStarted","Data":"3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5"} Sep 10 18:07:02.958504 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:02.958458 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:07:02.958727 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:02.958627 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Sep 10 18:07:02.958727 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:02.958671 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Sep 10 18:07:03.958014 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:03.957977 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Sep 10 18:07:03.958650 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:03.958616 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Sep 10 18:07:04.958127 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:04.958091 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Sep 10 18:07:04.958732 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:04.958703 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Sep 10 18:07:05.957889 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:05.957852 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Sep 10 18:07:05.958180 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:05.958150 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Sep 10 18:07:06.958132 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:06.957667 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Sep 10 18:07:06.958132 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:06.957731 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Sep 10 18:07:07.962628 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:07.962599 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:07:07.963065 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:07.962938 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:07:07.963869 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:07.963844 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:07:09.714791 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:09.714754 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:10.374291 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:10.374256 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:10.674627 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:10.674587 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:11.982105 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:11.982074 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:13.930381 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:13.930351 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:14.027731 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:14.027696 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:15.109074 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.109039 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:15.214404 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.214365 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:15.306785 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.306748 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:15.690797 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.690768 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:15.793169 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.793136 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:15.847943 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.847913 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb"] Sep 10 18:07:15.850268 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.850250 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:15.853606 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.853579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-4d9405-dockercfg-rrh2f\"" Sep 10 18:07:15.854380 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.854363 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-4d9405\"" Sep 10 18:07:15.865321 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.865293 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb"] Sep 10 18:07:15.908322 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.908300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683211de-3e11-483c-b77b-0f560b425c6c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:15.908436 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:15.908356 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtqkk\" (UniqueName: \"kubernetes.io/projected/683211de-3e11-483c-b77b-0f560b425c6c-kube-api-access-gtqkk\") pod \"payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:16.009158 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:16.009079 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtqkk\" (UniqueName: \"kubernetes.io/projected/683211de-3e11-483c-b77b-0f560b425c6c-kube-api-access-gtqkk\") pod \"payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:16.009331 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:16.009200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683211de-3e11-483c-b77b-0f560b425c6c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:16.009879 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:16.009859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683211de-3e11-483c-b77b-0f560b425c6c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:16.017985 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:16.017965 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtqkk\" (UniqueName: \"kubernetes.io/projected/683211de-3e11-483c-b77b-0f560b425c6c-kube-api-access-gtqkk\") pod \"payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:16.160303 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:16.160244 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:16.432699 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:16.432670 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb"] Sep 10 18:07:16.433702 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:07:16.433680 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683211de_3e11_483c_b77b_0f560b425c6c.slice/crio-1ebf714ee5815ca2715467cf0bc16ad7ed18b8ba2484c52d0658a5a5074290ae WatchSource:0}: Error finding container 1ebf714ee5815ca2715467cf0bc16ad7ed18b8ba2484c52d0658a5a5074290ae: Status 404 returned error can't find the container with id 1ebf714ee5815ca2715467cf0bc16ad7ed18b8ba2484c52d0658a5a5074290ae Sep 10 18:07:17.076885 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:17.076837 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" event={"ID":"683211de-3e11-483c-b77b-0f560b425c6c","Type":"ContainerStarted","Data":"112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52"} Sep 10 18:07:17.077090 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:17.076912 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" event={"ID":"683211de-3e11-483c-b77b-0f560b425c6c","Type":"ContainerStarted","Data":"1ebf714ee5815ca2715467cf0bc16ad7ed18b8ba2484c52d0658a5a5074290ae"} Sep 10 18:07:17.077176 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:17.077091 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:17.098350 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:17.098303 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" podStartSLOduration=2.098288873 podStartE2EDuration="2.098288873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:07:17.09660812 +0000 UTC m=+1565.978333263" watchObservedRunningTime="2026-09-10 18:07:17.098288873 +0000 UTC m=+1565.980014013" Sep 10 18:07:28.084386 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:28.084357 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:07:38.824514 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:38.824452 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:07:43.934553 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:43.934522 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb_683211de-3e11-483c-b77b-0f560b425c6c/payload-processing/0.log" Sep 10 18:07:58.934842 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:07:58.934810 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:11.849257 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:11.849167 2565 scope.go:117] "RemoveContainer" containerID="08409a0f5688f7a368d769939a041d1030008c2d9b4d1960a16b9e5de2f7302d" Sep 10 18:08:19.447784 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:19.447752 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m_1da3a56f-9e34-4794-86b9-ebd064e75b0e/payload-processing/0.log" Sep 10 18:08:19.677589 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:19.677558 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb_683211de-3e11-483c-b77b-0f560b425c6c/payload-processing/0.log" Sep 10 18:08:19.938530 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:19.938495 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:20.217500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.217418 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll"] Sep 10 18:08:20.219965 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.219950 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.224046 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.224024 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-3b8881-openshift-default-dockercfg-p84j5\"" Sep 10 18:08:20.234744 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.234715 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll"] Sep 10 18:08:20.236208 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.236181 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:20.377871 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.377842 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:20.396284 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-socket\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396420 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj8c8\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-kube-api-access-wj8c8\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396420 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396313 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-data\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396420 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-envoy\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396560 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396442 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-podinfo\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396560 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396494 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-token\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-certs\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddac52be-1649-4fae-ad93-935c896cfb5f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.396672 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.396650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-credential-socket\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.497899 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.497807 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-data\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.497899 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.497853 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-envoy\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.497967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-podinfo\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498033 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-token\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498084 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-certs\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddac52be-1649-4fae-ad93-935c896cfb5f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-credential-socket\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-socket\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8c8\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-kube-api-access-wj8c8\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498365 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-data\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498812 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498445 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-certs\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498812 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498650 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-credential-socket\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498812 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498757 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-socket\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.498812 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.498784 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddac52be-1649-4fae-ad93-935c896cfb5f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.500783 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.500757 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-envoy\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.500902 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.500869 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-podinfo\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.507895 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.507871 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-token\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.508406 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.508389 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8c8\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-kube-api-access-wj8c8\") pod \"e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.530513 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.530472 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:20.808756 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.808731 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll"] Sep 10 18:08:20.810378 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:08:20.810351 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddac52be_1649_4fae_ad93_935c896cfb5f.slice/crio-016ba6682550ed83e340f8bcb8b8bed9c87f2bd92fbd03ff211744a45cbba09a WatchSource:0}: Error finding container 016ba6682550ed83e340f8bcb8b8bed9c87f2bd92fbd03ff211744a45cbba09a: Status 404 returned error can't find the container with id 016ba6682550ed83e340f8bcb8b8bed9c87f2bd92fbd03ff211744a45cbba09a Sep 10 18:08:20.812648 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.812614 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:08:20.812727 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.812676 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:08:20.812727 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:20.812704 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:08:21.432384 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:21.432347 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" event={"ID":"ddac52be-1649-4fae-ad93-935c896cfb5f","Type":"ContainerStarted","Data":"a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23"} Sep 10 18:08:21.432587 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:21.432425 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" event={"ID":"ddac52be-1649-4fae-ad93-935c896cfb5f","Type":"ContainerStarted","Data":"016ba6682550ed83e340f8bcb8b8bed9c87f2bd92fbd03ff211744a45cbba09a"} Sep 10 18:08:21.460178 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:21.460128 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podStartSLOduration=1.460113162 podStartE2EDuration="1.460113162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:08:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:08:21.457290284 +0000 UTC m=+1630.339015439" watchObservedRunningTime="2026-09-10 18:08:21.460113162 +0000 UTC m=+1630.341838303" Sep 10 18:08:21.531366 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:21.531340 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:21.536479 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:21.536459 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:22.376379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:22.376342 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:22.438051 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:22.438031 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:22.438999 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:22.438981 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:23.365216 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:23.365179 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:23.449435 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:23.449398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:24.540321 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:24.540285 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:24.639941 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:24.639905 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:24.740943 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:24.740902 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:25.353263 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.353201 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:25.447870 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.447839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:25.641004 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.640925 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l"] Sep 10 18:08:25.645338 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.645314 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.664934 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.664886 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l"] Sep 10 18:08:25.666599 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.666573 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-3b8881-dockercfg-m7ndm\"" Sep 10 18:08:25.672784 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.671766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db1d357f-531d-4b7f-ab2a-0000df42f660-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.673085 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.673059 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k8dl\" (UniqueName: \"kubernetes.io/projected/db1d357f-531d-4b7f-ab2a-0000df42f660-kube-api-access-2k8dl\") pod \"payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.677108 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.677083 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-3b8881\"" Sep 10 18:08:25.773839 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.773810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db1d357f-531d-4b7f-ab2a-0000df42f660-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.773966 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.773864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k8dl\" (UniqueName: \"kubernetes.io/projected/db1d357f-531d-4b7f-ab2a-0000df42f660-kube-api-access-2k8dl\") pod \"payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.774506 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.774485 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db1d357f-531d-4b7f-ab2a-0000df42f660-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.789389 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.784694 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k8dl\" (UniqueName: \"kubernetes.io/projected/db1d357f-531d-4b7f-ab2a-0000df42f660-kube-api-access-2k8dl\") pod \"payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:25.986837 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:25.986807 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:26.259179 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:26.259154 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l"] Sep 10 18:08:26.260850 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:08:26.260821 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1d357f_531d_4b7f_ab2a_0000df42f660.slice/crio-6686e62e6adb97dd0df8cb409aaebd8e328dc930246e5ceab4b4dccebd56612a WatchSource:0}: Error finding container 6686e62e6adb97dd0df8cb409aaebd8e328dc930246e5ceab4b4dccebd56612a: Status 404 returned error can't find the container with id 6686e62e6adb97dd0df8cb409aaebd8e328dc930246e5ceab4b4dccebd56612a Sep 10 18:08:26.462504 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:26.462465 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" event={"ID":"db1d357f-531d-4b7f-ab2a-0000df42f660","Type":"ContainerStarted","Data":"49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57"} Sep 10 18:08:26.462692 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:26.462528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" event={"ID":"db1d357f-531d-4b7f-ab2a-0000df42f660","Type":"ContainerStarted","Data":"6686e62e6adb97dd0df8cb409aaebd8e328dc930246e5ceab4b4dccebd56612a"} Sep 10 18:08:26.462692 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:26.462557 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:26.514415 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:26.514297 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" podStartSLOduration=1.514271948 podStartE2EDuration="1.514271948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:08:26.511447839 +0000 UTC m=+1635.393173058" watchObservedRunningTime="2026-09-10 18:08:26.514271948 +0000 UTC m=+1635.395997090" Sep 10 18:08:37.475247 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:37.474607 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:44.572061 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:44.572012 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerID="a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23" exitCode=0 Sep 10 18:08:44.572509 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:44.572088 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" event={"ID":"ddac52be-1649-4fae-ad93-935c896cfb5f","Type":"ContainerDied","Data":"a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23"} Sep 10 18:08:44.572555 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:44.572545 2565 scope.go:117] "RemoveContainer" containerID="a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23" Sep 10 18:08:44.573720 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:44.573690 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:08:44.573829 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:44.573760 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:08:44.573829 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:44.573794 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 18:08:45.580727 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:45.580684 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" event={"ID":"ddac52be-1649-4fae-ad93-935c896cfb5f","Type":"ContainerStarted","Data":"73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af"} Sep 10 18:08:46.530827 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:46.530793 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:46.531009 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:46.530975 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Sep 10 18:08:46.531090 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:46.531061 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Sep 10 18:08:47.531379 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:47.531342 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Sep 10 18:08:47.531850 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:47.531419 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Sep 10 18:08:48.286751 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.286713 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:48.530842 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.530795 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Sep 10 18:08:48.531023 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.530868 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Sep 10 18:08:48.942004 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.941966 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc"] Sep 10 18:08:48.944653 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.944632 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:48.947316 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.947295 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:08:48.947422 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.947299 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:08:48.952940 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:48.952907 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc"] Sep 10 18:08:49.004210 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.004175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0318dbed-7268-4626-9e97-94418be45cd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc\" (UID: \"0318dbed-7268-4626-9e97-94418be45cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:49.105801 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.105772 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0318dbed-7268-4626-9e97-94418be45cd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc\" (UID: \"0318dbed-7268-4626-9e97-94418be45cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:49.106437 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.106415 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0318dbed-7268-4626-9e97-94418be45cd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc\" (UID: \"0318dbed-7268-4626-9e97-94418be45cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:49.258623 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.258527 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:49.531182 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.531102 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Sep 10 18:08:49.531182 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.531155 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Sep 10 18:08:49.538267 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.538238 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc"] Sep 10 18:08:49.539271 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:08:49.539245 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0318dbed_7268_4626_9e97_94418be45cd7.slice/crio-2fc96b1a3b11331e910594f06d4387623dfba3ad4f404cf1e19a52026ff0fab8 WatchSource:0}: Error finding container 2fc96b1a3b11331e910594f06d4387623dfba3ad4f404cf1e19a52026ff0fab8: Status 404 returned error can't find the container with id 2fc96b1a3b11331e910594f06d4387623dfba3ad4f404cf1e19a52026ff0fab8 Sep 10 18:08:49.611442 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:49.611407 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" event={"ID":"0318dbed-7268-4626-9e97-94418be45cd7","Type":"ContainerStarted","Data":"2fc96b1a3b11331e910594f06d4387623dfba3ad4f404cf1e19a52026ff0fab8"} Sep 10 18:08:50.531008 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:50.530973 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Sep 10 18:08:50.531413 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:50.531031 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Sep 10 18:08:50.619587 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:50.619549 2565 generic.go:356] "Generic (PLEG): container finished" podID="0318dbed-7268-4626-9e97-94418be45cd7" containerID="77cac212a6a8c21f333491964680b200ab4797a672ab0a6c6a8cfb492ab5f241" exitCode=0 Sep 10 18:08:50.619751 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:50.619636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" event={"ID":"0318dbed-7268-4626-9e97-94418be45cd7","Type":"ContainerDied","Data":"77cac212a6a8c21f333491964680b200ab4797a672ab0a6c6a8cfb492ab5f241"} Sep 10 18:08:51.535605 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.535576 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:51.536056 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.535857 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:51.536578 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.536557 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:08:51.771541 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.771520 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:51.832318 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.832258 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0318dbed-7268-4626-9e97-94418be45cd7-maas-api-service-ca\") pod \"0318dbed-7268-4626-9e97-94418be45cd7\" (UID: \"0318dbed-7268-4626-9e97-94418be45cd7\") " Sep 10 18:08:51.832570 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.832551 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0318dbed-7268-4626-9e97-94418be45cd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0318dbed-7268-4626-9e97-94418be45cd7" (UID: "0318dbed-7268-4626-9e97-94418be45cd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:08:51.933134 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:51.933102 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0318dbed-7268-4626-9e97-94418be45cd7-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:08:52.634362 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:52.634325 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" Sep 10 18:08:52.634762 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:52.634377 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc" event={"ID":"0318dbed-7268-4626-9e97-94418be45cd7","Type":"ContainerDied","Data":"2fc96b1a3b11331e910594f06d4387623dfba3ad4f404cf1e19a52026ff0fab8"} Sep 10 18:08:52.634762 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:52.634427 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fc96b1a3b11331e910594f06d4387623dfba3ad4f404cf1e19a52026ff0fab8" Sep 10 18:08:55.169648 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.169614 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:55.200654 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.200622 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l"] Sep 10 18:08:55.200857 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.200829 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" podUID="db1d357f-531d-4b7f-ab2a-0000df42f660" containerName="payload-processing" containerID="cri-o://49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57" gracePeriod=30 Sep 10 18:08:55.231407 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.231379 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:55.329356 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.329325 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:55.372580 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.372509 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc"] Sep 10 18:08:55.376271 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.376244 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-3b8881-wm7lc"] Sep 10 18:08:55.387582 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.387554 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:55.456354 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.456324 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:55.564843 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.564806 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db1d357f-531d-4b7f-ab2a-0000df42f660-plugins-config-volume\") pod \"db1d357f-531d-4b7f-ab2a-0000df42f660\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " Sep 10 18:08:55.565019 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.564912 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k8dl\" (UniqueName: \"kubernetes.io/projected/db1d357f-531d-4b7f-ab2a-0000df42f660-kube-api-access-2k8dl\") pod \"db1d357f-531d-4b7f-ab2a-0000df42f660\" (UID: \"db1d357f-531d-4b7f-ab2a-0000df42f660\") " Sep 10 18:08:55.565272 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.565208 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db1d357f-531d-4b7f-ab2a-0000df42f660-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db1d357f-531d-4b7f-ab2a-0000df42f660" (UID: "db1d357f-531d-4b7f-ab2a-0000df42f660"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:08:55.565371 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.565357 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db1d357f-531d-4b7f-ab2a-0000df42f660-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:08:55.566984 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.566956 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1d357f-531d-4b7f-ab2a-0000df42f660-kube-api-access-2k8dl" (OuterVolumeSpecName: "kube-api-access-2k8dl") pod "db1d357f-531d-4b7f-ab2a-0000df42f660" (UID: "db1d357f-531d-4b7f-ab2a-0000df42f660"). InnerVolumeSpecName "kube-api-access-2k8dl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:08:55.623484 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.623449 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0318dbed-7268-4626-9e97-94418be45cd7" path="/var/lib/kubelet/pods/0318dbed-7268-4626-9e97-94418be45cd7/volumes" Sep 10 18:08:55.651305 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.651277 2565 generic.go:356] "Generic (PLEG): container finished" podID="db1d357f-531d-4b7f-ab2a-0000df42f660" containerID="49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57" exitCode=0 Sep 10 18:08:55.651400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.651309 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" event={"ID":"db1d357f-531d-4b7f-ab2a-0000df42f660","Type":"ContainerDied","Data":"49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57"} Sep 10 18:08:55.651400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.651325 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" Sep 10 18:08:55.651400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.651359 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l" event={"ID":"db1d357f-531d-4b7f-ab2a-0000df42f660","Type":"ContainerDied","Data":"6686e62e6adb97dd0df8cb409aaebd8e328dc930246e5ceab4b4dccebd56612a"} Sep 10 18:08:55.651400 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.651386 2565 scope.go:117] "RemoveContainer" containerID="49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57" Sep 10 18:08:55.660345 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.660326 2565 scope.go:117] "RemoveContainer" containerID="49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57" Sep 10 18:08:55.660591 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:08:55.660570 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57\": container with ID starting with 49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57 not found: ID does not exist" containerID="49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57" Sep 10 18:08:55.660647 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.660598 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57"} err="failed to get container status \"49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57\": rpc error: code = NotFound desc = could not find container \"49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57\": container with ID starting with 49d501774e3b09aed2f7bfd1db46ae172ff7ba700fc0ec4a199318d6d40e8b57 not found: ID does not exist" Sep 10 18:08:55.666154 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.666135 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k8dl\" (UniqueName: \"kubernetes.io/projected/db1d357f-531d-4b7f-ab2a-0000df42f660-kube-api-access-2k8dl\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:08:55.669825 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.669795 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l"] Sep 10 18:08:55.671920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.671891 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3b8881-5c6d5c698c-cgz5l"] Sep 10 18:08:55.716518 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:55.716464 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:08:57.624491 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:08:57.624458 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db1d357f-531d-4b7f-ab2a-0000df42f660" path="/var/lib/kubelet/pods/db1d357f-531d-4b7f-ab2a-0000df42f660/volumes" Sep 10 18:09:01.821381 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:01.821339 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:01.922536 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:01.922505 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:02.055863 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:02.055825 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll"] Sep 10 18:09:02.056141 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:02.056113 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" containerID="cri-o://73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af" gracePeriod=30 Sep 10 18:09:02.068712 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:02.068686 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:03.387852 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:03.387816 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:05.505308 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505275 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj"] Sep 10 18:09:05.505718 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505702 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1d357f-531d-4b7f-ab2a-0000df42f660" containerName="payload-processing" Sep 10 18:09:05.505718 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505718 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1d357f-531d-4b7f-ab2a-0000df42f660" containerName="payload-processing" Sep 10 18:09:05.505798 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505731 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0318dbed-7268-4626-9e97-94418be45cd7" containerName="revoke-keys" Sep 10 18:09:05.505798 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505737 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0318dbed-7268-4626-9e97-94418be45cd7" containerName="revoke-keys" Sep 10 18:09:05.505858 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505825 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0318dbed-7268-4626-9e97-94418be45cd7" containerName="revoke-keys" Sep 10 18:09:05.505858 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.505836 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1d357f-531d-4b7f-ab2a-0000df42f660" containerName="payload-processing" Sep 10 18:09:05.508717 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.508700 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:05.511378 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.511354 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:09:05.511488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.511462 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:09:05.516463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.516435 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj"] Sep 10 18:09:05.660764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.660733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97999ba1-f09f-476a-a199-1081121db820-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-732455-465pj\" (UID: \"97999ba1-f09f-476a-a199-1081121db820\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:05.724155 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.724111 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:05.762174 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.762098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97999ba1-f09f-476a-a199-1081121db820-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-732455-465pj\" (UID: \"97999ba1-f09f-476a-a199-1081121db820\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:05.762776 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.762749 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97999ba1-f09f-476a-a199-1081121db820-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-732455-465pj\" (UID: \"97999ba1-f09f-476a-a199-1081121db820\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:05.821910 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:05.821880 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:06.090334 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:06.090310 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj"] Sep 10 18:09:06.092670 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:09:06.092647 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97999ba1_f09f_476a_a199_1081121db820.slice/crio-7874341546a8636a866615336c7eaadc640ed12394d83c90b0fd2de89af4236b WatchSource:0}: Error finding container 7874341546a8636a866615336c7eaadc640ed12394d83c90b0fd2de89af4236b: Status 404 returned error can't find the container with id 7874341546a8636a866615336c7eaadc640ed12394d83c90b0fd2de89af4236b Sep 10 18:09:06.538540 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:06.538510 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 18:09:06.538862 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:06.538566 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 18:09:06.715471 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:06.715433 2565 generic.go:356] "Generic (PLEG): container finished" podID="97999ba1-f09f-476a-a199-1081121db820" containerID="b96879320fa4069fabd7a3d6c5627c297b6d0f1747b065afe4989227d65a14bc" exitCode=0 Sep 10 18:09:06.715600 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:06.715513 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" event={"ID":"97999ba1-f09f-476a-a199-1081121db820","Type":"ContainerDied","Data":"b96879320fa4069fabd7a3d6c5627c297b6d0f1747b065afe4989227d65a14bc"} Sep 10 18:09:06.715600 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:06.715556 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" event={"ID":"97999ba1-f09f-476a-a199-1081121db820","Type":"ContainerStarted","Data":"7874341546a8636a866615336c7eaadc640ed12394d83c90b0fd2de89af4236b"} Sep 10 18:09:07.311021 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.310993 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:09:07.480796 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.480768 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-certs\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.480964 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.480810 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-data\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.480964 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.480852 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-envoy\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.480964 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.480887 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-podinfo\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.480964 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.480947 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj8c8\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-kube-api-access-wj8c8\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.481185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.480969 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-token\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.481185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481014 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:09:07.481185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481026 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddac52be-1649-4fae-ad93-935c896cfb5f-istiod-ca-cert\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.481185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481105 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:09:07.481185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481141 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-credential-socket\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.481185 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481181 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-socket\") pod \"ddac52be-1649-4fae-ad93-935c896cfb5f\" (UID: \"ddac52be-1649-4fae-ad93-935c896cfb5f\") " Sep 10 18:09:07.481535 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481413 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:09:07.481535 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481465 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddac52be-1649-4fae-ad93-935c896cfb5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:09:07.481645 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481619 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:09:07.481783 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481761 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddac52be-1649-4fae-ad93-935c896cfb5f-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.481845 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481788 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.481845 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481804 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.481845 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481817 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.481845 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.481831 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.483391 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.483353 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:09:07.483500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.483372 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:09:07.483500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.483384 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-kube-api-access-wj8c8" (OuterVolumeSpecName: "kube-api-access-wj8c8") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "kube-api-access-wj8c8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:09:07.483500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.483457 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ddac52be-1649-4fae-ad93-935c896cfb5f" (UID: "ddac52be-1649-4fae-ad93-935c896cfb5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:09:07.583268 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.583216 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.583268 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.583267 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.583592 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.583281 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj8c8\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-kube-api-access-wj8c8\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.583592 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.583294 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddac52be-1649-4fae-ad93-935c896cfb5f-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:07.722773 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.722739 2565 generic.go:356] "Generic (PLEG): container finished" podID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerID="73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af" exitCode=0 Sep 10 18:09:07.722920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.722786 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" Sep 10 18:09:07.722920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.722779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" event={"ID":"ddac52be-1649-4fae-ad93-935c896cfb5f","Type":"ContainerDied","Data":"73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af"} Sep 10 18:09:07.722920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.722904 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll" event={"ID":"ddac52be-1649-4fae-ad93-935c896cfb5f","Type":"ContainerDied","Data":"016ba6682550ed83e340f8bcb8b8bed9c87f2bd92fbd03ff211744a45cbba09a"} Sep 10 18:09:07.723074 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.722929 2565 scope.go:117] "RemoveContainer" containerID="73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af" Sep 10 18:09:07.732632 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.732617 2565 scope.go:117] "RemoveContainer" containerID="a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23" Sep 10 18:09:07.741669 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.741644 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll"] Sep 10 18:09:07.743662 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.743569 2565 scope.go:117] "RemoveContainer" containerID="73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af" Sep 10 18:09:07.744539 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:09:07.744515 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af\": container with ID starting with 73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af not found: ID does not exist" containerID="73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af" Sep 10 18:09:07.744646 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.744548 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af"} err="failed to get container status \"73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af\": rpc error: code = NotFound desc = could not find container \"73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af\": container with ID starting with 73ccc0523aedf42738cb7ce82f8c5e11751a2322c0839a44ded5505d6872b1af not found: ID does not exist" Sep 10 18:09:07.744646 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.744584 2565 scope.go:117] "RemoveContainer" containerID="a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23" Sep 10 18:09:07.744909 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:09:07.744887 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23\": container with ID starting with a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23 not found: ID does not exist" containerID="a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23" Sep 10 18:09:07.744981 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.744916 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23"} err="failed to get container status \"a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23\": rpc error: code = NotFound desc = could not find container \"a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23\": container with ID starting with a83ffe8afa5e9879e6bf65a45bec3d2f76c40a405ae12df018f0efd4bc8b6f23 not found: ID does not exist" Sep 10 18:09:07.746729 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.746711 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b8881-openshift-default-784c596-v7vll"] Sep 10 18:09:07.843429 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.843404 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:07.986386 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.986294 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97999ba1-f09f-476a-a199-1081121db820-maas-api-service-ca\") pod \"97999ba1-f09f-476a-a199-1081121db820\" (UID: \"97999ba1-f09f-476a-a199-1081121db820\") " Sep 10 18:09:07.986687 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:07.986660 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97999ba1-f09f-476a-a199-1081121db820-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97999ba1-f09f-476a-a199-1081121db820" (UID: "97999ba1-f09f-476a-a199-1081121db820"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:09:08.087878 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:08.087852 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97999ba1-f09f-476a-a199-1081121db820-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:08.729823 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:08.729791 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" Sep 10 18:09:08.730252 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:08.729825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj" event={"ID":"97999ba1-f09f-476a-a199-1081121db820","Type":"ContainerDied","Data":"7874341546a8636a866615336c7eaadc640ed12394d83c90b0fd2de89af4236b"} Sep 10 18:09:08.730252 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:08.729879 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7874341546a8636a866615336c7eaadc640ed12394d83c90b0fd2de89af4236b" Sep 10 18:09:09.623321 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:09.623280 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" path="/var/lib/kubelet/pods/ddac52be-1649-4fae-ad93-935c896cfb5f/volumes" Sep 10 18:09:21.079463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.079417 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:21.104687 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.104655 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m"] Sep 10 18:09:21.104930 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.104908 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" podUID="1da3a56f-9e34-4794-86b9-ebd064e75b0e" containerName="payload-processing" containerID="cri-o://62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd" gracePeriod=30 Sep 10 18:09:21.112856 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.112540 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt"] Sep 10 18:09:21.112856 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.112763 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" podUID="06af970f-b41a-43aa-96fe-edba191c52a7" containerName="payload-pre-processing" containerID="cri-o://f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f" gracePeriod=30 Sep 10 18:09:21.147928 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.147895 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:21.206883 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.206853 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:21.277473 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.277411 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj"] Sep 10 18:09:21.279032 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.278882 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:21.281522 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.281495 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-732455-465pj"] Sep 10 18:09:21.339617 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.339543 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:21.376956 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.376932 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:09:21.388444 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.388427 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:09:21.409282 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.409262 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sltb9\" (UniqueName: \"kubernetes.io/projected/06af970f-b41a-43aa-96fe-edba191c52a7-kube-api-access-sltb9\") pod \"06af970f-b41a-43aa-96fe-edba191c52a7\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " Sep 10 18:09:21.409388 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.409323 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da3a56f-9e34-4794-86b9-ebd064e75b0e-plugins-config-volume\") pod \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " Sep 10 18:09:21.409463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.409388 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jhmc\" (UniqueName: \"kubernetes.io/projected/1da3a56f-9e34-4794-86b9-ebd064e75b0e-kube-api-access-4jhmc\") pod \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\" (UID: \"1da3a56f-9e34-4794-86b9-ebd064e75b0e\") " Sep 10 18:09:21.409463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.409425 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06af970f-b41a-43aa-96fe-edba191c52a7-plugins-config-volume\") pod \"06af970f-b41a-43aa-96fe-edba191c52a7\" (UID: \"06af970f-b41a-43aa-96fe-edba191c52a7\") " Sep 10 18:09:21.409752 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.409729 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da3a56f-9e34-4794-86b9-ebd064e75b0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1da3a56f-9e34-4794-86b9-ebd064e75b0e" (UID: "1da3a56f-9e34-4794-86b9-ebd064e75b0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:09:21.409834 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.409799 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06af970f-b41a-43aa-96fe-edba191c52a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06af970f-b41a-43aa-96fe-edba191c52a7" (UID: "06af970f-b41a-43aa-96fe-edba191c52a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:09:21.411323 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.411304 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06af970f-b41a-43aa-96fe-edba191c52a7-kube-api-access-sltb9" (OuterVolumeSpecName: "kube-api-access-sltb9") pod "06af970f-b41a-43aa-96fe-edba191c52a7" (UID: "06af970f-b41a-43aa-96fe-edba191c52a7"). InnerVolumeSpecName "kube-api-access-sltb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:09:21.411449 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.411423 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da3a56f-9e34-4794-86b9-ebd064e75b0e-kube-api-access-4jhmc" (OuterVolumeSpecName: "kube-api-access-4jhmc") pod "1da3a56f-9e34-4794-86b9-ebd064e75b0e" (UID: "1da3a56f-9e34-4794-86b9-ebd064e75b0e"). InnerVolumeSpecName "kube-api-access-4jhmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:09:21.510801 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.510768 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jhmc\" (UniqueName: \"kubernetes.io/projected/1da3a56f-9e34-4794-86b9-ebd064e75b0e-kube-api-access-4jhmc\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:21.510801 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.510795 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06af970f-b41a-43aa-96fe-edba191c52a7-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:21.510801 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.510804 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sltb9\" (UniqueName: \"kubernetes.io/projected/06af970f-b41a-43aa-96fe-edba191c52a7-kube-api-access-sltb9\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:21.511033 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.510813 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da3a56f-9e34-4794-86b9-ebd064e75b0e-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:09:21.623887 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.623816 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97999ba1-f09f-476a-a199-1081121db820" path="/var/lib/kubelet/pods/97999ba1-f09f-476a-a199-1081121db820/volumes" Sep 10 18:09:21.654003 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.653966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:21.806179 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.806150 2565 generic.go:356] "Generic (PLEG): container finished" podID="06af970f-b41a-43aa-96fe-edba191c52a7" containerID="f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f" exitCode=0 Sep 10 18:09:21.806307 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.806189 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" Sep 10 18:09:21.806307 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.806257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" event={"ID":"06af970f-b41a-43aa-96fe-edba191c52a7","Type":"ContainerDied","Data":"f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f"} Sep 10 18:09:21.806307 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.806303 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt" event={"ID":"06af970f-b41a-43aa-96fe-edba191c52a7","Type":"ContainerDied","Data":"4b8817677c181bdd78a1f0f877aa3a50c9387a391fb6648018a735ee12abf809"} Sep 10 18:09:21.806458 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.806328 2565 scope.go:117] "RemoveContainer" containerID="f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f" Sep 10 18:09:21.807741 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.807715 2565 generic.go:356] "Generic (PLEG): container finished" podID="1da3a56f-9e34-4794-86b9-ebd064e75b0e" containerID="62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd" exitCode=0 Sep 10 18:09:21.807854 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.807753 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" Sep 10 18:09:21.807854 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.807753 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" event={"ID":"1da3a56f-9e34-4794-86b9-ebd064e75b0e","Type":"ContainerDied","Data":"62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd"} Sep 10 18:09:21.807982 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.807860 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m" event={"ID":"1da3a56f-9e34-4794-86b9-ebd064e75b0e","Type":"ContainerDied","Data":"a564524d735880c190f7d4df9b4ec406e8d4d84bc50a129e1e01985b34381096"} Sep 10 18:09:21.815167 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.815147 2565 scope.go:117] "RemoveContainer" containerID="f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f" Sep 10 18:09:21.815435 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:09:21.815416 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f\": container with ID starting with f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f not found: ID does not exist" containerID="f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f" Sep 10 18:09:21.815508 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.815444 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f"} err="failed to get container status \"f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f\": rpc error: code = NotFound desc = could not find container \"f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f\": container with ID starting with f6851ecc08af42c84940d8d2edf2a0d5184a5202dc16769b754b7a4bb9b9689f not found: ID does not exist" Sep 10 18:09:21.815508 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.815460 2565 scope.go:117] "RemoveContainer" containerID="62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd" Sep 10 18:09:21.823768 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.823751 2565 scope.go:117] "RemoveContainer" containerID="62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd" Sep 10 18:09:21.824040 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:09:21.824019 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd\": container with ID starting with 62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd not found: ID does not exist" containerID="62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd" Sep 10 18:09:21.824109 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.824045 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd"} err="failed to get container status \"62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd\": rpc error: code = NotFound desc = could not find container \"62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd\": container with ID starting with 62b26ef0c53d4fdd6147ea3d2ec529601c2818d833c633ab47645be8ef6041fd not found: ID does not exist" Sep 10 18:09:21.825117 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.825098 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt"] Sep 10 18:09:21.832940 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.832921 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-732455-5c7ccffdf4-8htzt"] Sep 10 18:09:21.846694 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.846674 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m"] Sep 10 18:09:21.851008 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:21.850987 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-732455-7994c465d9-4kn7m"] Sep 10 18:09:23.623087 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:23.623055 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06af970f-b41a-43aa-96fe-edba191c52a7" path="/var/lib/kubelet/pods/06af970f-b41a-43aa-96fe-edba191c52a7/volumes" Sep 10 18:09:23.623486 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:23.623471 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da3a56f-9e34-4794-86b9-ebd064e75b0e" path="/var/lib/kubelet/pods/1da3a56f-9e34-4794-86b9-ebd064e75b0e/volumes" Sep 10 18:09:26.665184 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:26.665156 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:09:47.420279 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:09:47.420168 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:02.915872 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:02.915829 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:03.695180 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:03.695148 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:09.267255 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.267206 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:09.368461 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.368426 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:09.513491 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.513453 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq"] Sep 10 18:10:09.513791 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.513742 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" containerID="cri-o://3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5" gracePeriod=30 Sep 10 18:10:09.535635 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.535569 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:09.917118 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917033 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv"] Sep 10 18:10:09.917481 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917467 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06af970f-b41a-43aa-96fe-edba191c52a7" containerName="payload-pre-processing" Sep 10 18:10:09.917532 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917482 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="06af970f-b41a-43aa-96fe-edba191c52a7" containerName="payload-pre-processing" Sep 10 18:10:09.917532 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917506 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97999ba1-f09f-476a-a199-1081121db820" containerName="revoke-keys" Sep 10 18:10:09.917532 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917512 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="97999ba1-f09f-476a-a199-1081121db820" containerName="revoke-keys" Sep 10 18:10:09.917532 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917521 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da3a56f-9e34-4794-86b9-ebd064e75b0e" containerName="payload-processing" Sep 10 18:10:09.917532 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917532 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da3a56f-9e34-4794-86b9-ebd064e75b0e" containerName="payload-processing" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917541 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917547 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917555 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917562 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917636 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="97999ba1-f09f-476a-a199-1081121db820" containerName="revoke-keys" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917644 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917657 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da3a56f-9e34-4794-86b9-ebd064e75b0e" containerName="payload-processing" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917663 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddac52be-1649-4fae-ad93-935c896cfb5f" containerName="istio-proxy" Sep 10 18:10:09.917695 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.917670 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="06af970f-b41a-43aa-96fe-edba191c52a7" containerName="payload-pre-processing" Sep 10 18:10:09.920913 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.920894 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:09.924051 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.924017 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:10:09.924198 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.924099 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:10:09.937513 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.937484 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv"] Sep 10 18:10:09.974535 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:09.974509 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/060eac7d-db9c-4a81-ac22-fcf2cd806774-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv\" (UID: \"060eac7d-db9c-4a81-ac22-fcf2cd806774\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:10.075492 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:10.075464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/060eac7d-db9c-4a81-ac22-fcf2cd806774-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv\" (UID: \"060eac7d-db9c-4a81-ac22-fcf2cd806774\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:10.076022 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:10.075993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/060eac7d-db9c-4a81-ac22-fcf2cd806774-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv\" (UID: \"060eac7d-db9c-4a81-ac22-fcf2cd806774\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:10.139042 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:10.139000 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:10.233421 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:10.233380 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:10.503293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:10.503266 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv"] Sep 10 18:10:10.505457 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:10:10.505428 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060eac7d_db9c_4a81_ac22_fcf2cd806774.slice/crio-274b34bb4b943d7da14d9ab7f72ca1dca78f07ef0491fcba9940cceedefe3c01 WatchSource:0}: Error finding container 274b34bb4b943d7da14d9ab7f72ca1dca78f07ef0491fcba9940cceedefe3c01: Status 404 returned error can't find the container with id 274b34bb4b943d7da14d9ab7f72ca1dca78f07ef0491fcba9940cceedefe3c01 Sep 10 18:10:11.081524 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:11.081497 2565 generic.go:356] "Generic (PLEG): container finished" podID="060eac7d-db9c-4a81-ac22-fcf2cd806774" containerID="a7c05e99ab3bab7755db89e34ed8ec857bcb6a22649e2a0a193de611a5612862" exitCode=0 Sep 10 18:10:11.081659 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:11.081584 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" event={"ID":"060eac7d-db9c-4a81-ac22-fcf2cd806774","Type":"ContainerDied","Data":"a7c05e99ab3bab7755db89e34ed8ec857bcb6a22649e2a0a193de611a5612862"} Sep 10 18:10:11.081659 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:11.081640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" event={"ID":"060eac7d-db9c-4a81-ac22-fcf2cd806774","Type":"ContainerStarted","Data":"274b34bb4b943d7da14d9ab7f72ca1dca78f07ef0491fcba9940cceedefe3c01"} Sep 10 18:10:12.220870 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:12.220842 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:12.297959 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:12.297927 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/060eac7d-db9c-4a81-ac22-fcf2cd806774-maas-api-service-ca\") pod \"060eac7d-db9c-4a81-ac22-fcf2cd806774\" (UID: \"060eac7d-db9c-4a81-ac22-fcf2cd806774\") " Sep 10 18:10:12.298312 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:12.298290 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/060eac7d-db9c-4a81-ac22-fcf2cd806774-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "060eac7d-db9c-4a81-ac22-fcf2cd806774" (UID: "060eac7d-db9c-4a81-ac22-fcf2cd806774"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:10:12.398906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:12.398877 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/060eac7d-db9c-4a81-ac22-fcf2cd806774-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:13.092307 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:13.092273 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" Sep 10 18:10:13.092307 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:13.092298 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv" event={"ID":"060eac7d-db9c-4a81-ac22-fcf2cd806774","Type":"ContainerDied","Data":"274b34bb4b943d7da14d9ab7f72ca1dca78f07ef0491fcba9940cceedefe3c01"} Sep 10 18:10:13.092534 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:13.092346 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="274b34bb4b943d7da14d9ab7f72ca1dca78f07ef0491fcba9940cceedefe3c01" Sep 10 18:10:14.773823 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.773796 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:10:14.924302 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924267 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-token\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924490 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924332 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-podinfo\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924490 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924361 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-data\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924606 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924573 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-envoy\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924659 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924627 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tl92\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-kube-api-access-6tl92\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924659 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924651 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:10:14.924772 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924674 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-socket\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924772 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924734 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-certs\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924874 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924801 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-credential-socket\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.924874 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924858 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istiod-ca-cert\") pod \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\" (UID: \"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0\") " Sep 10 18:10:14.925007 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.924968 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:10:14.925166 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.925140 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:10:14.925295 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.925283 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:14.925358 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.925187 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:10:14.925358 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.925303 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:14.925446 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.925386 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:10:14.926608 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.926581 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:10:14.926608 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.926577 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:10:14.927080 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.927054 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:10:14.927080 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:14.927063 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-kube-api-access-6tl92" (OuterVolumeSpecName: "kube-api-access-6tl92") pod "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" (UID: "3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0"). InnerVolumeSpecName "kube-api-access-6tl92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:10:15.026548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026499 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.026548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026541 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.026548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026552 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.026548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026560 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.026548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026568 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.026849 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026575 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.026849 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.026584 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tl92\" (UniqueName: \"kubernetes.io/projected/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0-kube-api-access-6tl92\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:15.104747 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.104709 2565 generic.go:356] "Generic (PLEG): container finished" podID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerID="3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5" exitCode=0 Sep 10 18:10:15.104945 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.104785 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" Sep 10 18:10:15.104945 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.104795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" event={"ID":"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0","Type":"ContainerDied","Data":"3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5"} Sep 10 18:10:15.104945 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.104893 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq" event={"ID":"3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0","Type":"ContainerDied","Data":"498c9dbb4fb3c99664ddf5c526a8367a7ca7778b86bb38bb9f8c128be522dab9"} Sep 10 18:10:15.104945 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.104923 2565 scope.go:117] "RemoveContainer" containerID="3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5" Sep 10 18:10:15.115076 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.115057 2565 scope.go:117] "RemoveContainer" containerID="e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10" Sep 10 18:10:15.124010 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.123984 2565 scope.go:117] "RemoveContainer" containerID="3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5" Sep 10 18:10:15.124373 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:10:15.124352 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5\": container with ID starting with 3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5 not found: ID does not exist" containerID="3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5" Sep 10 18:10:15.124434 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.124384 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5"} err="failed to get container status \"3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5\": rpc error: code = NotFound desc = could not find container \"3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5\": container with ID starting with 3dbf42a134db0603b2d59a456510a435718826f668849107e73d5780d188ccc5 not found: ID does not exist" Sep 10 18:10:15.124434 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.124403 2565 scope.go:117] "RemoveContainer" containerID="e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10" Sep 10 18:10:15.124664 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:10:15.124649 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10\": container with ID starting with e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10 not found: ID does not exist" containerID="e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10" Sep 10 18:10:15.124705 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.124668 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10"} err="failed to get container status \"e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10\": rpc error: code = NotFound desc = could not find container \"e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10\": container with ID starting with e89655b04b67a0be19b8dc678e3cf8627b6dd7985f9094a863a2b66919ff5b10 not found: ID does not exist" Sep 10 18:10:15.128621 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.128595 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq"] Sep 10 18:10:15.132427 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.132404 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-732455-openshift-default-844dd7bcf4-qb5bq"] Sep 10 18:10:15.622944 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:15.622906 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" path="/var/lib/kubelet/pods/3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0/volumes" Sep 10 18:10:16.147967 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:16.147937 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:20.151213 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.151183 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:20.700967 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.700929 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:20.737680 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.737642 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb"] Sep 10 18:10:20.737917 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.737883 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" podUID="683211de-3e11-483c-b77b-0f560b425c6c" containerName="payload-processing" containerID="cri-o://112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52" gracePeriod=30 Sep 10 18:10:20.800414 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.800385 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:20.888311 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.886615 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv"] Sep 10 18:10:20.892082 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.892039 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4d9405-jfzvv"] Sep 10 18:10:20.893612 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.893581 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:20.947037 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.947004 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:20.999094 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:20.999068 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:10:21.086720 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.086691 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtqkk\" (UniqueName: \"kubernetes.io/projected/683211de-3e11-483c-b77b-0f560b425c6c-kube-api-access-gtqkk\") pod \"683211de-3e11-483c-b77b-0f560b425c6c\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " Sep 10 18:10:21.086898 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.086741 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683211de-3e11-483c-b77b-0f560b425c6c-plugins-config-volume\") pod \"683211de-3e11-483c-b77b-0f560b425c6c\" (UID: \"683211de-3e11-483c-b77b-0f560b425c6c\") " Sep 10 18:10:21.087144 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.087119 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/683211de-3e11-483c-b77b-0f560b425c6c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "683211de-3e11-483c-b77b-0f560b425c6c" (UID: "683211de-3e11-483c-b77b-0f560b425c6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:10:21.088720 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.088698 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683211de-3e11-483c-b77b-0f560b425c6c-kube-api-access-gtqkk" (OuterVolumeSpecName: "kube-api-access-gtqkk") pod "683211de-3e11-483c-b77b-0f560b425c6c" (UID: "683211de-3e11-483c-b77b-0f560b425c6c"). InnerVolumeSpecName "kube-api-access-gtqkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:10:21.139925 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.139898 2565 generic.go:356] "Generic (PLEG): container finished" podID="683211de-3e11-483c-b77b-0f560b425c6c" containerID="112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52" exitCode=0 Sep 10 18:10:21.140040 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.139947 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" Sep 10 18:10:21.140040 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.139984 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" event={"ID":"683211de-3e11-483c-b77b-0f560b425c6c","Type":"ContainerDied","Data":"112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52"} Sep 10 18:10:21.140112 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.140077 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb" event={"ID":"683211de-3e11-483c-b77b-0f560b425c6c","Type":"ContainerDied","Data":"1ebf714ee5815ca2715467cf0bc16ad7ed18b8ba2484c52d0658a5a5074290ae"} Sep 10 18:10:21.140112 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.140105 2565 scope.go:117] "RemoveContainer" containerID="112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52" Sep 10 18:10:21.150051 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.150034 2565 scope.go:117] "RemoveContainer" containerID="112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52" Sep 10 18:10:21.150370 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:10:21.150347 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52\": container with ID starting with 112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52 not found: ID does not exist" containerID="112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52" Sep 10 18:10:21.150448 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.150377 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52"} err="failed to get container status \"112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52\": rpc error: code = NotFound desc = could not find container \"112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52\": container with ID starting with 112eef077169910d8b33816cc8c4946d2b111d81ce6f3e79cc92e139e5bd0e52 not found: ID does not exist" Sep 10 18:10:21.163549 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.163520 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb"] Sep 10 18:10:21.167493 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.167470 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4d9405-769f89bddc-dfgzb"] Sep 10 18:10:21.187874 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.187851 2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683211de-3e11-483c-b77b-0f560b425c6c-plugins-config-volume\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:21.187874 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.187872 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtqkk\" (UniqueName: \"kubernetes.io/projected/683211de-3e11-483c-b77b-0f560b425c6c-kube-api-access-gtqkk\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:21.247283 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.247185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:21.623053 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.622971 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="060eac7d-db9c-4a81-ac22-fcf2cd806774" path="/var/lib/kubelet/pods/060eac7d-db9c-4a81-ac22-fcf2cd806774/volumes" Sep 10 18:10:21.623373 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:21.623359 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683211de-3e11-483c-b77b-0f560b425c6c" path="/var/lib/kubelet/pods/683211de-3e11-483c-b77b-0f560b425c6c/volumes" Sep 10 18:10:27.213195 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.213161 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:27.314113 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.314079 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:27.499890 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.499812 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:27.816392 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816311 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw"] Sep 10 18:10:27.816766 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816748 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" Sep 10 18:10:27.816857 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816768 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" Sep 10 18:10:27.816857 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816797 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683211de-3e11-483c-b77b-0f560b425c6c" containerName="payload-processing" Sep 10 18:10:27.816857 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816806 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="683211de-3e11-483c-b77b-0f560b425c6c" containerName="payload-processing" Sep 10 18:10:27.816857 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816829 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="060eac7d-db9c-4a81-ac22-fcf2cd806774" containerName="revoke-keys" Sep 10 18:10:27.816857 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816837 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="060eac7d-db9c-4a81-ac22-fcf2cd806774" containerName="revoke-keys" Sep 10 18:10:27.817114 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816944 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" Sep 10 18:10:27.817114 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816959 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="060eac7d-db9c-4a81-ac22-fcf2cd806774" containerName="revoke-keys" Sep 10 18:10:27.817114 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816969 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" Sep 10 18:10:27.817114 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.816977 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="683211de-3e11-483c-b77b-0f560b425c6c" containerName="payload-processing" Sep 10 18:10:27.821606 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.821586 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:27.824425 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.824407 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:10:27.824527 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.824436 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:10:27.830207 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.830183 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw"] Sep 10 18:10:27.949625 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:27.949596 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e7f5fc7-83f6-4f22-8954-5b0403b54d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw\" (UID: \"6e7f5fc7-83f6-4f22-8954-5b0403b54d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:28.005516 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:28.005488 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:28.050627 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:28.050604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e7f5fc7-83f6-4f22-8954-5b0403b54d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw\" (UID: \"6e7f5fc7-83f6-4f22-8954-5b0403b54d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:28.051188 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:28.051170 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e7f5fc7-83f6-4f22-8954-5b0403b54d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw\" (UID: \"6e7f5fc7-83f6-4f22-8954-5b0403b54d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:28.133548 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:28.133460 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:28.406572 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:28.406546 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw"] Sep 10 18:10:28.408505 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:10:28.408475 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7f5fc7_83f6_4f22_8954_5b0403b54d47.slice/crio-7c462b0756727c05fff338d51c50b56f51372124f160cdc83f0f57df4b325979 WatchSource:0}: Error finding container 7c462b0756727c05fff338d51c50b56f51372124f160cdc83f0f57df4b325979: Status 404 returned error can't find the container with id 7c462b0756727c05fff338d51c50b56f51372124f160cdc83f0f57df4b325979 Sep 10 18:10:29.192709 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:29.192680 2565 generic.go:356] "Generic (PLEG): container finished" podID="6e7f5fc7-83f6-4f22-8954-5b0403b54d47" containerID="d9e2bd6efc6dd43bde656d877ec20d0ebafd9614c17ef2e807eb1871cfbe3814" exitCode=0 Sep 10 18:10:29.192832 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:29.192769 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" event={"ID":"6e7f5fc7-83f6-4f22-8954-5b0403b54d47","Type":"ContainerDied","Data":"d9e2bd6efc6dd43bde656d877ec20d0ebafd9614c17ef2e807eb1871cfbe3814"} Sep 10 18:10:29.192832 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:29.192824 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" event={"ID":"6e7f5fc7-83f6-4f22-8954-5b0403b54d47","Type":"ContainerStarted","Data":"7c462b0756727c05fff338d51c50b56f51372124f160cdc83f0f57df4b325979"} Sep 10 18:10:30.333155 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:30.333134 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:30.471488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:30.471424 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e7f5fc7-83f6-4f22-8954-5b0403b54d47-maas-api-service-ca\") pod \"6e7f5fc7-83f6-4f22-8954-5b0403b54d47\" (UID: \"6e7f5fc7-83f6-4f22-8954-5b0403b54d47\") " Sep 10 18:10:30.471744 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:30.471722 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e7f5fc7-83f6-4f22-8954-5b0403b54d47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e7f5fc7-83f6-4f22-8954-5b0403b54d47" (UID: "6e7f5fc7-83f6-4f22-8954-5b0403b54d47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:10:30.572735 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:30.572710 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e7f5fc7-83f6-4f22-8954-5b0403b54d47-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:31.206367 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:31.206339 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" Sep 10 18:10:31.206534 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:31.206378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw" event={"ID":"6e7f5fc7-83f6-4f22-8954-5b0403b54d47","Type":"ContainerDied","Data":"7c462b0756727c05fff338d51c50b56f51372124f160cdc83f0f57df4b325979"} Sep 10 18:10:31.206534 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:31.206424 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c462b0756727c05fff338d51c50b56f51372124f160cdc83f0f57df4b325979" Sep 10 18:10:38.271741 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.271706 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:38.279800 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.279772 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc"] Sep 10 18:10:38.281828 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.281805 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6c9ffc-29817720-zmpgc"] Sep 10 18:10:38.333854 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.333815 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:38.435624 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.435586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:38.458931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.458902 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw"] Sep 10 18:10:38.461536 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.461514 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6c9ffc-fq5gw"] Sep 10 18:10:38.501811 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.501785 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:38.828509 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:38.828467 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:39.622820 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:39.622784 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e36c5fc-5b5c-44e4-91ec-a04051036159" path="/var/lib/kubelet/pods/4e36c5fc-5b5c-44e4-91ec-a04051036159/volumes" Sep 10 18:10:39.623337 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:39.623244 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e7f5fc7-83f6-4f22-8954-5b0403b54d47" path="/var/lib/kubelet/pods/6e7f5fc7-83f6-4f22-8954-5b0403b54d47/volumes" Sep 10 18:10:45.051272 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.051209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:45.103050 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.103009 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:45.342304 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.342201 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:45.665885 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.665807 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd"] Sep 10 18:10:45.666214 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.666200 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e7f5fc7-83f6-4f22-8954-5b0403b54d47" containerName="revoke-keys" Sep 10 18:10:45.666293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.666216 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e7f5fc7-83f6-4f22-8954-5b0403b54d47" containerName="revoke-keys" Sep 10 18:10:45.666293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.666254 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" Sep 10 18:10:45.666293 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.666262 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0d68cb-24fd-47d8-9e6d-e9c19c5974a0" containerName="istio-proxy" Sep 10 18:10:45.666390 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.666351 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e7f5fc7-83f6-4f22-8954-5b0403b54d47" containerName="revoke-keys" Sep 10 18:10:45.669436 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.669420 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:45.672119 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.672098 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22shs\"" Sep 10 18:10:45.672251 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.672165 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 18:10:45.678888 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.678860 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd"] Sep 10 18:10:45.714692 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.714667 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15dc9e0f-22a8-463b-89b7-ca87010db82d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd\" (UID: \"15dc9e0f-22a8-463b-89b7-ca87010db82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:45.815878 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.815839 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15dc9e0f-22a8-463b-89b7-ca87010db82d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd\" (UID: \"15dc9e0f-22a8-463b-89b7-ca87010db82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:45.816488 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.816465 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15dc9e0f-22a8-463b-89b7-ca87010db82d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd\" (UID: \"15dc9e0f-22a8-463b-89b7-ca87010db82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:45.929423 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.929398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:45.980903 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:45.980882 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:46.253931 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:46.253903 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd"] Sep 10 18:10:46.256060 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:10:46.256031 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15dc9e0f_22a8_463b_89b7_ca87010db82d.slice/crio-6bceba286d7ceda5f03fe687f3df57198e1085e7016b08ba25253d8d96fcdde1 WatchSource:0}: Error finding container 6bceba286d7ceda5f03fe687f3df57198e1085e7016b08ba25253d8d96fcdde1: Status 404 returned error can't find the container with id 6bceba286d7ceda5f03fe687f3df57198e1085e7016b08ba25253d8d96fcdde1 Sep 10 18:10:46.292859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:46.292833 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" event={"ID":"15dc9e0f-22a8-463b-89b7-ca87010db82d","Type":"ContainerStarted","Data":"6bceba286d7ceda5f03fe687f3df57198e1085e7016b08ba25253d8d96fcdde1"} Sep 10 18:10:47.300407 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:47.300327 2565 generic.go:356] "Generic (PLEG): container finished" podID="15dc9e0f-22a8-463b-89b7-ca87010db82d" containerID="abe4d03d91607ae2446fcd809ecf6e3398fa2607b5e003f12d2cd504a9cfe3ae" exitCode=0 Sep 10 18:10:47.300764 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:47.300414 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" event={"ID":"15dc9e0f-22a8-463b-89b7-ca87010db82d","Type":"ContainerDied","Data":"abe4d03d91607ae2446fcd809ecf6e3398fa2607b5e003f12d2cd504a9cfe3ae"} Sep 10 18:10:48.430657 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:48.430633 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:48.539490 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:48.539464 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15dc9e0f-22a8-463b-89b7-ca87010db82d-maas-api-service-ca\") pod \"15dc9e0f-22a8-463b-89b7-ca87010db82d\" (UID: \"15dc9e0f-22a8-463b-89b7-ca87010db82d\") " Sep 10 18:10:48.539784 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:48.539763 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15dc9e0f-22a8-463b-89b7-ca87010db82d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15dc9e0f-22a8-463b-89b7-ca87010db82d" (UID: "15dc9e0f-22a8-463b-89b7-ca87010db82d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:10:48.640399 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:48.640324 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15dc9e0f-22a8-463b-89b7-ca87010db82d-maas-api-service-ca\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:10:49.312729 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:49.312692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" event={"ID":"15dc9e0f-22a8-463b-89b7-ca87010db82d","Type":"ContainerDied","Data":"6bceba286d7ceda5f03fe687f3df57198e1085e7016b08ba25253d8d96fcdde1"} Sep 10 18:10:49.312729 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:49.312731 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd" Sep 10 18:10:49.312939 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:49.312746 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bceba286d7ceda5f03fe687f3df57198e1085e7016b08ba25253d8d96fcdde1" Sep 10 18:10:56.247358 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.247302 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr"] Sep 10 18:10:56.249207 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.249185 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc23cf-29817720-zhfcr"] Sep 10 18:10:56.253247 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.253201 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:56.349547 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.349511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:56.428842 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.428803 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd"] Sep 10 18:10:56.431477 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.431429 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:56.432446 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.432429 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc23cf-fqqrd"] Sep 10 18:10:56.494459 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.494429 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:56.799617 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:56.799583 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:10:57.622541 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:57.622503 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15dc9e0f-22a8-463b-89b7-ca87010db82d" path="/var/lib/kubelet/pods/15dc9e0f-22a8-463b-89b7-ca87010db82d/volumes" Sep 10 18:10:57.622916 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:10:57.622873 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6481e1d-d625-4945-a5bf-9f3aaae7fff2" path="/var/lib/kubelet/pods/b6481e1d-d625-4945-a5bf-9f3aaae7fff2/volumes" Sep 10 18:11:03.108087 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:03.108051 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:11:03.160622 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:03.160586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:11:03.373774 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:03.373695 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5"] Sep 10 18:11:03.373960 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:03.373937 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" containerID="cri-o://95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6" gracePeriod=30 Sep 10 18:11:03.383007 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:03.382978 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:11:08.648651 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.648583 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 18:11:08.839332 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839295 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-envoy\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839332 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839335 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4mmc\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-kube-api-access-c4mmc\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839359 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-credential-socket\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839373 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-token\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839431 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-certs\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839449 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-podinfo\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839473 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-socket\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839511 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-data\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839584 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839584 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb047b72-2959-4dcc-953d-41e41b4d298b-istiod-ca-cert\") pod \"eb047b72-2959-4dcc-953d-41e41b4d298b\" (UID: \"eb047b72-2959-4dcc-953d-41e41b4d298b\") " Sep 10 18:11:08.839955 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839683 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:11:08.839955 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839744 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:11:08.839955 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839881 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:11:08.840128 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.839983 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb047b72-2959-4dcc-953d-41e41b4d298b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:11:08.840128 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.840001 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-credential-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.840128 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.840029 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-certs\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.840128 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.840043 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-workload-socket\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.840128 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.840082 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:11:08.841553 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.841523 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:11:08.841689 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.841667 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-kube-api-access-c4mmc" (OuterVolumeSpecName: "kube-api-access-c4mmc") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "kube-api-access-c4mmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:11:08.842000 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.841977 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 18:11:08.842056 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.841989 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb047b72-2959-4dcc-953d-41e41b4d298b" (UID: "eb047b72-2959-4dcc-953d-41e41b4d298b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:11:08.941328 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.941302 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-podinfo\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.941328 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.941324 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-data\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.941328 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.941333 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb047b72-2959-4dcc-953d-41e41b4d298b-istiod-ca-cert\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.941525 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.941341 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-envoy\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.941525 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.941350 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4mmc\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-kube-api-access-c4mmc\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:08.941525 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:08.941359 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb047b72-2959-4dcc-953d-41e41b4d298b-istio-token\") on node \"ip-10-0-134-57.ec2.internal\" DevicePath \"\"" Sep 10 18:11:09.433914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.433883 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerID="95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6" exitCode=0 Sep 10 18:11:09.434149 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.433930 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" event={"ID":"eb047b72-2959-4dcc-953d-41e41b4d298b","Type":"ContainerDied","Data":"95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6"} Sep 10 18:11:09.434149 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.433942 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" Sep 10 18:11:09.434149 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.434019 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5" event={"ID":"eb047b72-2959-4dcc-953d-41e41b4d298b","Type":"ContainerDied","Data":"ad707090e46929c5cf78eab7f4bec1fdf972ca1ecd9275c62be4ebf6a3abdab6"} Sep 10 18:11:09.434149 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.434051 2565 scope.go:117] "RemoveContainer" containerID="95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6" Sep 10 18:11:09.444772 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.444748 2565 scope.go:117] "RemoveContainer" containerID="36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d" Sep 10 18:11:09.455401 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.455315 2565 scope.go:117] "RemoveContainer" containerID="95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6" Sep 10 18:11:09.455934 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:11:09.455905 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6\": container with ID starting with 95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6 not found: ID does not exist" containerID="95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6" Sep 10 18:11:09.456058 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.455944 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6"} err="failed to get container status \"95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6\": rpc error: code = NotFound desc = could not find container \"95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6\": container with ID starting with 95429d70f649b5be99b43270b0596c448945d882ea5b78e05a657207ed7265f6 not found: ID does not exist" Sep 10 18:11:09.456058 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.455963 2565 scope.go:117] "RemoveContainer" containerID="36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d" Sep 10 18:11:09.456332 ip-10-0-134-57 kubenswrapper[2565]: E0910 18:11:09.456307 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d\": container with ID starting with 36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d not found: ID does not exist" containerID="36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d" Sep 10 18:11:09.456438 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.456335 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d"} err="failed to get container status \"36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d\": rpc error: code = NotFound desc = could not find container \"36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d\": container with ID starting with 36b7d523c732abb9a697be9b43ead923e65748b5386a3f4eb78376a2044d508d not found: ID does not exist" Sep 10 18:11:09.458725 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.458695 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5"] Sep 10 18:11:09.460555 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.460534 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fc23cf-openshift-default-659dd6c4d4-s4vs5"] Sep 10 18:11:09.623415 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:09.623382 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" path="/var/lib/kubelet/pods/eb047b72-2959-4dcc-953d-41e41b4d298b/volumes" Sep 10 18:11:11.904106 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:11.904074 2565 scope.go:117] "RemoveContainer" containerID="867513d0e56282f5907786f7fc6e022f4178a4106ce040ce6e88a10d28466709" Sep 10 18:11:11.912612 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:11.912582 2565 scope.go:117] "RemoveContainer" containerID="07db3464d60bbfb87ebd57df4c022b98a03c389173de9297cc28d8854368d4d7" Sep 10 18:11:11.920425 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:11.920406 2565 scope.go:117] "RemoveContainer" containerID="612a3a2b72a7e74fa3d5b96d273b7247305d62e709a117c72b057ba3f18d8ce2" Sep 10 18:11:22.630479 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:22.630439 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:11:22.689765 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:11:22.689723 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:12:11.960756 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:12:11.960717 2565 scope.go:117] "RemoveContainer" containerID="6b97801eed81dc232d2c9729255255aaf3b634a4a109c1f9ab9465bc3c8effd7" Sep 10 18:12:11.969355 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:12:11.969330 2565 scope.go:117] "RemoveContainer" containerID="517f94e8f53603307f4bc51c505dd1734b63ee724a58a79c39c9913f019cdc9e" Sep 10 18:12:28.845321 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:12:28.845279 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:12:31.068239 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:12:31.068194 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:12:32.579279 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:12:32.579247 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:13:11.994335 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:13:11.994298 2565 scope.go:117] "RemoveContainer" containerID="ef77e001f5518ef8b7ffdec034bf8e25dbc5d6cfbbcbe255cf5b2de46f6df749" Sep 10 18:13:37.513211 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:13:37.513127 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:14:38.543145 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:14:38.543062 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:14:56.653209 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:14:56.653172 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:15:12.021708 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:12.021671 2565 scope.go:117] "RemoveContainer" containerID="b96879320fa4069fabd7a3d6c5627c297b6d0f1747b065afe4989227d65a14bc" Sep 10 18:15:12.030201 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:12.030182 2565 scope.go:117] "RemoveContainer" containerID="77cac212a6a8c21f333491964680b200ab4797a672ab0a6c6a8cfb492ab5f241" Sep 10 18:15:24.580856 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:24.580817 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:15:26.494009 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:26.493973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:15:31.711753 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:31.711717 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:15:56.560136 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:56.560101 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:15:57.993507 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:15:57.993473 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:16:12.052896 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:12.052856 2565 scope.go:117] "RemoveContainer" containerID="a7c05e99ab3bab7755db89e34ed8ec857bcb6a22649e2a0a193de611a5612862" Sep 10 18:16:55.715944 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:55.715907 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gnb4w_c4e74032-3bc2-4023-9c9c-23961ef64df3/manager/0.log" Sep 10 18:16:56.085371 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:56.085287 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-xkr99_1330a45b-d686-4705-9035-8f9bba0f01ba/copy-manifests/0.log" Sep 10 18:16:56.091919 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:56.091892 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-xkr99_1330a45b-d686-4705-9035-8f9bba0f01ba/manager/0.log" Sep 10 18:16:58.726685 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:58.726652 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5_e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce/istio-proxy/0.log" Sep 10 18:16:58.848482 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:58.848443 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-brd5r_74e78f6a-ba52-4e5a-bb21-9aaeb1072565/discovery/0.log" Sep 10 18:16:59.206217 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:59.206175 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/1.log" Sep 10 18:16:59.327322 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:59.327258 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ncr7j_c347e8ae-f065-4b20-a978-129098798e99/payload-pre-processing/0.log" Sep 10 18:16:59.576592 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:59.576485 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765484f768-8d5m8_511026f7-b2e0-47ae-8d79-51eab0beb87a/router/0.log" Sep 10 18:16:59.915829 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:16:59.915751 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bcbl7_eb58e274-72a5-4529-973c-8404c8d731f1/main/0.log" Sep 10 18:17:00.028650 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:00.028617 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wslb2_3407dc1e-fc6a-48be-a623-d0baa0902cec/main/0.log" Sep 10 18:17:00.475613 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:00.475580 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rkr59_70e42ecd-0b24-4128-9e88-b546c7ce0ad0/main/0.log" Sep 10 18:17:06.380091 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:06.379995 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:17:06.434541 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:06.434502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" limit=1024 Sep 10 18:17:09.426115 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:09.426078 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vqz5s_2cd31ec3-6041-40ea-a2cd-a991b38024c7/global-pull-secret-syncer/0.log" Sep 10 18:17:09.469463 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:09.469421 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6rs8k_cbc90ac2-79a2-42fe-8212-55b616354d80/konnectivity-agent/0.log" Sep 10 18:17:09.550818 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:09.550791 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-57.ec2.internal_c6e324fa94971b9656e873f1a2a8f1c1/haproxy/0.log" Sep 10 18:17:12.073443 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:12.073408 2565 scope.go:117] "RemoveContainer" containerID="d9e2bd6efc6dd43bde656d877ec20d0ebafd9614c17ef2e807eb1871cfbe3814" Sep 10 18:17:12.082137 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:12.082111 2565 scope.go:117] "RemoveContainer" containerID="abe4d03d91607ae2446fcd809ecf6e3398fa2607b5e003f12d2cd504a9cfe3ae" Sep 10 18:17:16.348521 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.348493 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/alertmanager/0.log" Sep 10 18:17:16.367176 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.367147 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/config-reloader/0.log" Sep 10 18:17:16.386831 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.386798 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/kube-rbac-proxy-web/0.log" Sep 10 18:17:16.404057 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.404027 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/kube-rbac-proxy/0.log" Sep 10 18:17:16.421544 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.421517 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/kube-rbac-proxy-metric/0.log" Sep 10 18:17:16.442001 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.441968 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/prom-label-proxy/0.log" Sep 10 18:17:16.459151 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.459119 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b2617bc1-9b0a-41fe-9709-25680cfac182/init-config-reloader/0.log" Sep 10 18:17:16.492778 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.492744 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-lzggl_5958b543-bd6a-4291-b525-944cc570ea22/cluster-monitoring-operator/0.log" Sep 10 18:17:16.514626 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.514594 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-mwkjz_1d2fbff6-084b-4725-9b96-276231f3c073/kube-state-metrics/0.log" Sep 10 18:17:16.529472 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.529442 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-mwkjz_1d2fbff6-084b-4725-9b96-276231f3c073/kube-rbac-proxy-main/0.log" Sep 10 18:17:16.546826 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.546791 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-mwkjz_1d2fbff6-084b-4725-9b96-276231f3c073/kube-rbac-proxy-self/0.log" Sep 10 18:17:16.591361 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.591329 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-jkjb6_ee3970c1-46d6-4f00-b183-916b1778a572/monitoring-plugin/0.log" Sep 10 18:17:16.753534 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.753500 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k9sl7_d248c12c-b9d4-458b-8182-48b0c12a3303/node-exporter/0.log" Sep 10 18:17:16.790487 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.790455 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k9sl7_d248c12c-b9d4-458b-8182-48b0c12a3303/kube-rbac-proxy/0.log" Sep 10 18:17:16.808303 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.808276 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k9sl7_d248c12c-b9d4-458b-8182-48b0c12a3303/init-textfile/0.log" Sep 10 18:17:16.845596 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.845563 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-g6zvv_75f70879-f9cd-4ea4-b9c8-518e071f5c5b/kube-rbac-proxy-main/0.log" Sep 10 18:17:16.866929 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.866895 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-g6zvv_75f70879-f9cd-4ea4-b9c8-518e071f5c5b/kube-rbac-proxy-self/0.log" Sep 10 18:17:16.886591 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:16.886563 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-g6zvv_75f70879-f9cd-4ea4-b9c8-518e071f5c5b/openshift-state-metrics/0.log" Sep 10 18:17:17.154897 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:17.154813 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c8b855fcd-g9k2p_9d6784c3-02a8-4cea-ac8c-b52a40d91477/telemeter-client/0.log" Sep 10 18:17:17.175712 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:17.175683 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c8b855fcd-g9k2p_9d6784c3-02a8-4cea-ac8c-b52a40d91477/reload/0.log" Sep 10 18:17:17.193526 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:17.193497 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c8b855fcd-g9k2p_9d6784c3-02a8-4cea-ac8c-b52a40d91477/kube-rbac-proxy/0.log" Sep 10 18:17:18.756500 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:18.756468 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-vdf6l_e06c3493-58ed-4a5a-b1f0-a9bd8d673ead/networking-console-plugin/0.log" Sep 10 18:17:19.883799 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:19.883764 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7465887497-dcj9v_bfcf0ac4-3ea1-474b-9902-c97a9d15dfca/console/0.log" Sep 10 18:17:20.123878 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.123834 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq"] Sep 10 18:17:20.124355 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124336 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" Sep 10 18:17:20.124355 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124356 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" Sep 10 18:17:20.124459 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124371 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15dc9e0f-22a8-463b-89b7-ca87010db82d" containerName="revoke-keys" Sep 10 18:17:20.124459 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124377 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="15dc9e0f-22a8-463b-89b7-ca87010db82d" containerName="revoke-keys" Sep 10 18:17:20.124459 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124445 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="15dc9e0f-22a8-463b-89b7-ca87010db82d" containerName="revoke-keys" Sep 10 18:17:20.124459 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124454 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" Sep 10 18:17:20.124590 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.124461 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb047b72-2959-4dcc-953d-41e41b4d298b" containerName="istio-proxy" Sep 10 18:17:20.127340 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.127318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.130280 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.130247 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h9jz8\"/\"openshift-service-ca.crt\"" Sep 10 18:17:20.130424 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.130251 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h9jz8\"/\"kube-root-ca.crt\"" Sep 10 18:17:20.131169 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.131147 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h9jz8\"/\"default-dockercfg-zgjz4\"" Sep 10 18:17:20.136746 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.136718 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq"] Sep 10 18:17:20.200622 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.200582 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-proc\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.200622 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.200628 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-lib-modules\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.200920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.200650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qbv7\" (UniqueName: \"kubernetes.io/projected/16e04fdd-6033-4a57-b9d4-ba4e200ca765-kube-api-access-9qbv7\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.200920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.200819 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-podres\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.200920 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.200870 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-sys\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.301669 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qbv7\" (UniqueName: \"kubernetes.io/projected/16e04fdd-6033-4a57-b9d4-ba4e200ca765-kube-api-access-9qbv7\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.301859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301722 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-podres\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.301859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-sys\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.301859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-proc\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.301859 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-lib-modules\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.302031 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301868 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-sys\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.302031 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301893 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-podres\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.302031 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301880 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-proc\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.302031 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.301931 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16e04fdd-6033-4a57-b9d4-ba4e200ca765-lib-modules\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.309894 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.309857 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qbv7\" (UniqueName: \"kubernetes.io/projected/16e04fdd-6033-4a57-b9d4-ba4e200ca765-kube-api-access-9qbv7\") pod \"perf-node-gather-daemonset-7z9wq\" (UID: \"16e04fdd-6033-4a57-b9d4-ba4e200ca765\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.439288 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.439221 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:20.674489 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.674455 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-cc8xm_b7d62301-70d4-4c47-b170-e7b5dcebae76/volume-data-source-validator/0.log" Sep 10 18:17:20.712945 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:20.712915 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq"] Sep 10 18:17:20.715464 ip-10-0-134-57 kubenswrapper[2565]: W0910 18:17:20.715433 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod16e04fdd_6033_4a57_b9d4_ba4e200ca765.slice/crio-b2a51e5a3fc4f92809074693a1f6fb3324b19c9b43fadb20de4f8f9c78456cd6 WatchSource:0}: Error finding container b2a51e5a3fc4f92809074693a1f6fb3324b19c9b43fadb20de4f8f9c78456cd6: Status 404 returned error can't find the container with id b2a51e5a3fc4f92809074693a1f6fb3324b19c9b43fadb20de4f8f9c78456cd6 Sep 10 18:17:21.383758 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.383720 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" event={"ID":"16e04fdd-6033-4a57-b9d4-ba4e200ca765","Type":"ContainerStarted","Data":"6a0022e8f9c821b3296ba4776763588fe55c34148998c4d807581dea7bbedbe5"} Sep 10 18:17:21.384167 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.383776 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" event={"ID":"16e04fdd-6033-4a57-b9d4-ba4e200ca765","Type":"ContainerStarted","Data":"b2a51e5a3fc4f92809074693a1f6fb3324b19c9b43fadb20de4f8f9c78456cd6"} Sep 10 18:17:21.384167 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.383798 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:21.399925 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.399869 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" podStartSLOduration=1.39985308 podStartE2EDuration="1.39985308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:17:21.398143718 +0000 UTC m=+2170.279868862" watchObservedRunningTime="2026-09-10 18:17:21.39985308 +0000 UTC m=+2170.281578263" Sep 10 18:17:21.812766 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.812734 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6tl2g_a1cde6b8-5c75-434a-8880-1db69cfd7ebe/dns/0.log" Sep 10 18:17:21.829881 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.829846 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6tl2g_a1cde6b8-5c75-434a-8880-1db69cfd7ebe/kube-rbac-proxy/0.log" Sep 10 18:17:21.926109 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:21.926078 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6b7qr_79157419-0d17-4cb0-8ebe-e1efbb0fd39c/dns-node-resolver/0.log" Sep 10 18:17:22.528787 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:22.528760 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mh7pm_067eb756-a003-43b7-8693-116193f3ae2c/node-ca/0.log" Sep 10 18:17:23.393098 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:23.393059 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5_e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce/istio-proxy/0.log" Sep 10 18:17:23.618572 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:23.618534 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-brd5r_74e78f6a-ba52-4e5a-bb21-9aaeb1072565/discovery/0.log" Sep 10 18:17:23.685351 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:23.685317 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/0.log" Sep 10 18:17:23.693347 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:23.693316 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/1.log" Sep 10 18:17:23.723433 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:23.723393 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ncr7j_c347e8ae-f065-4b20-a978-129098798e99/payload-pre-processing/0.log" Sep 10 18:17:23.782378 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:23.782342 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765484f768-8d5m8_511026f7-b2e0-47ae-8d79-51eab0beb87a/router/0.log" Sep 10 18:17:24.614924 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:24.614889 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5tvgn_94aed250-dd5b-4c9c-aeb5-9178dde79af0/serve-healthcheck-canary/0.log" Sep 10 18:17:25.190730 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:25.190699 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktl4g_ae938dde-0a01-4dc8-a034-10b6b953f26a/kube-rbac-proxy/0.log" Sep 10 18:17:25.207909 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:25.207880 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktl4g_ae938dde-0a01-4dc8-a034-10b6b953f26a/exporter/0.log" Sep 10 18:17:25.224766 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:25.224736 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktl4g_ae938dde-0a01-4dc8-a034-10b6b953f26a/extractor/0.log" Sep 10 18:17:27.401093 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:27.401060 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-7z9wq" Sep 10 18:17:27.696075 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:27.696037 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gnb4w_c4e74032-3bc2-4023-9c9c-23961ef64df3/manager/0.log" Sep 10 18:17:27.848079 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:27.848024 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-xkr99_1330a45b-d686-4705-9035-8f9bba0f01ba/manager/0.log" Sep 10 18:17:27.864616 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:27.864589 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-xkr99_1330a45b-d686-4705-9035-8f9bba0f01ba/copy-manifests/0.log" Sep 10 18:17:29.135504 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:29.135472 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-746f98bbcc-kn9qx_ffc090a4-ca7d-4ded-b01e-017e6f81cad4/manager/0.log" Sep 10 18:17:30.969198 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:30.969168 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5_e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce/istio-proxy/0.log" Sep 10 18:17:31.195657 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:31.195623 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-brd5r_74e78f6a-ba52-4e5a-bb21-9aaeb1072565/discovery/0.log" Sep 10 18:17:31.245368 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:31.245274 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/0.log" Sep 10 18:17:31.253413 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:31.253381 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/1.log" Sep 10 18:17:31.276914 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:31.276879 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ncr7j_c347e8ae-f065-4b20-a978-129098798e99/payload-pre-processing/0.log" Sep 10 18:17:31.329367 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:31.329331 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765484f768-8d5m8_511026f7-b2e0-47ae-8d79-51eab0beb87a/router/0.log" Sep 10 18:17:33.013906 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:33.013875 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cnnsl5_e2696ef9-6d15-4bab-9e50-dd1b4b9a7dce/istio-proxy/0.log" Sep 10 18:17:33.241483 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:33.241443 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-brd5r_74e78f6a-ba52-4e5a-bb21-9aaeb1072565/discovery/0.log" Sep 10 18:17:33.293517 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:33.293423 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/0.log" Sep 10 18:17:33.301414 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:33.301382 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-n7pc7_ac2ec7ce-2b19-4a7e-ba20-f88f490d804e/istio-proxy/1.log" Sep 10 18:17:33.323786 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:33.323743 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ncr7j_c347e8ae-f065-4b20-a978-129098798e99/payload-pre-processing/0.log" Sep 10 18:17:33.376666 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:33.376631 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-765484f768-8d5m8_511026f7-b2e0-47ae-8d79-51eab0beb87a/router/0.log" Sep 10 18:17:34.897899 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:34.897866 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pnqlv_de119af3-a57d-41b9-a334-63c626cc1d27/kube-storage-version-migrator-operator/1.log" Sep 10 18:17:34.898731 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:34.898711 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pnqlv_de119af3-a57d-41b9-a334-63c626cc1d27/kube-storage-version-migrator-operator/0.log" Sep 10 18:17:36.090946 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.090910 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/kube-multus-additional-cni-plugins/0.log" Sep 10 18:17:36.108814 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.108788 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/egress-router-binary-copy/0.log" Sep 10 18:17:36.125780 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.125756 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/cni-plugins/0.log" Sep 10 18:17:36.142254 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.142203 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/bond-cni-plugin/0.log" Sep 10 18:17:36.158477 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.158447 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/routeoverride-cni/0.log" Sep 10 18:17:36.175218 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.175188 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/whereabouts-cni-bincopy/0.log" Sep 10 18:17:36.191306 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.191278 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9wd6j_fd061ead-6c8f-42ce-b26a-29377ea343a0/whereabouts-cni/0.log" Sep 10 18:17:36.311540 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.311506 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zjrg9_e0aa3af8-0964-41e7-9863-552d00698063/kube-multus/0.log" Sep 10 18:17:36.369746 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.369664 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g2jbw_d9588d65-3514-42ee-9252-56457aeeac79/network-metrics-daemon/0.log" Sep 10 18:17:36.387199 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:36.387169 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g2jbw_d9588d65-3514-42ee-9252-56457aeeac79/kube-rbac-proxy/0.log" Sep 10 18:17:37.219301 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.219262 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/ovn-controller/0.log" Sep 10 18:17:37.233157 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.233129 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/ovn-acl-logging/0.log" Sep 10 18:17:37.252739 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.252712 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/kube-rbac-proxy-node/0.log" Sep 10 18:17:37.269994 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.269957 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 18:17:37.286713 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.286684 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/northd/0.log" Sep 10 18:17:37.302728 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.302689 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/nbdb/0.log" Sep 10 18:17:37.319067 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.319032 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/sbdb/0.log" Sep 10 18:17:37.370905 ip-10-0-134-57 kubenswrapper[2565]: I0910 18:17:37.370870 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22nhh_98733f1c-e61e-48e1-8025-38d5a558d6c8/ovnkube-controller/0.log"