Sep 06 19:25:43.453639 ip-10-0-135-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 06 19:25:43.453651 ip-10-0-135-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 06 19:25:43.453660 ip-10-0-135-236 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 06 19:25:43.454020 ip-10-0-135-236 systemd[1]: Failed to start Kubernetes Kubelet. Sep 06 19:25:53.581909 ip-10-0-135-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 06 19:25:53.581931 ip-10-0-135-236 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 902ebd139e494918ac3c23847759a671 -- Sep 06 19:27:37.148010 ip-10-0-135-236 systemd[1]: Starting Kubernetes Kubelet... Sep 06 19:27:37.531634 ip-10-0-135-236 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 19:27:37.531634 ip-10-0-135-236 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 06 19:27:37.531634 ip-10-0-135-236 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 19:27:37.531634 ip-10-0-135-236 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 06 19:27:37.531634 ip-10-0-135-236 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 19:27:37.532349 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.531700 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 06 19:27:37.537456 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537440 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 19:27:37.537456 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537456 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537460 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537463 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537468 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537471 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537475 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537477 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537480 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537483 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537486 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537489 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537491 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537494 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537497 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537499 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537502 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537511 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537514 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537518 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 19:27:37.537528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537522 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537525 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537528 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537530 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537533 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537536 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537539 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537542 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537544 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537547 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537549 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537552 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537554 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537557 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537559 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537562 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537564 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537566 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537569 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537572 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 19:27:37.538007 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537574 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537577 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537579 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537582 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537585 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537587 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537615 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537620 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537623 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537627 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537630 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537632 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537635 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537638 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537640 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537643 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537646 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537648 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537651 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537653 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 19:27:37.538489 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537656 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537659 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537662 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537664 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537667 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537670 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537672 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537675 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537677 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537680 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537682 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537685 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537687 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537691 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537695 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537698 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537701 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537703 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537706 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 19:27:37.538993 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537709 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537712 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537714 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537717 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537720 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537722 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537726 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537728 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537731 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537733 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537736 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537738 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537741 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537743 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537761 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537765 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537769 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537773 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537776 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 19:27:37.539487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.537779 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 19:27:37.540123 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540111 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 19:27:37.540123 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540123 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540126 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540129 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540131 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540135 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540139 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540143 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540145 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540148 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540151 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540153 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540156 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540159 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540161 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540164 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540167 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540170 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540172 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540175 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540178 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 19:27:37.540178 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540180 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540183 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540186 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540189 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540191 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540194 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540196 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540199 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540201 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540204 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540206 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540209 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540211 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540214 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540216 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540219 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540221 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540224 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540226 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540229 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 19:27:37.540668 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540232 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540234 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540237 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540239 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540242 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540244 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540247 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540249 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540252 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540254 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540257 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540259 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540262 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540264 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540268 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540272 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540274 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540277 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540279 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 19:27:37.541322 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540284 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540287 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540290 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540293 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540296 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540300 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540302 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540305 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540308 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540310 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540313 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540316 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540318 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540322 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540324 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540327 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540329 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540332 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540334 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 19:27:37.542025 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540337 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540339 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540342 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540345 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540347 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540350 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540352 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540355 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540358 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540361 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540363 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540366 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540368 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540371 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540373 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540376 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540378 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540380 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540383 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 19:27:37.542500 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.540385 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 19:27:37.543143 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543128 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 19:27:37.543143 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543143 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543147 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543150 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543154 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543157 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543160 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543163 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543165 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543168 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543170 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543173 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543176 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543179 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543182 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543185 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543189 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543193 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543196 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543199 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543202 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 19:27:37.543201 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543205 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543208 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543215 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543218 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543221 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543224 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543227 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543229 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543232 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543235 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543238 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543240 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543243 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543246 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543248 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543252 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543255 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543257 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543260 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 19:27:37.543671 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543262 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543264 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543267 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543271 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543275 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543278 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543281 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543284 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543287 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543290 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543292 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543295 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543298 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543301 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543304 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543307 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543311 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543313 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543318 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543322 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 19:27:37.544156 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543324 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543327 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543329 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543332 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543334 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543337 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543339 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543342 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543344 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543347 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543350 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543352 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543355 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543357 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543360 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543363 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543365 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543368 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543370 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543373 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 19:27:37.544682 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543376 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543378 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543381 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543384 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543387 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543390 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543393 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543396 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543398 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543401 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543403 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543406 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543408 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543411 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543413 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543416 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543418 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543421 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.543423 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543495 2575 flags.go:64] FLAG: --address="0.0.0.0" Sep 06 19:27:37.545179 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543539 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543547 2575 flags.go:64] FLAG: --anonymous-auth="true" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543551 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543556 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543559 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543563 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543568 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543571 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543574 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543577 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543581 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543584 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543587 2575 flags.go:64] FLAG: --cgroup-root="" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543590 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543593 2575 flags.go:64] FLAG: --client-ca-file="" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543596 2575 flags.go:64] FLAG: --cloud-provider="external" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543600 2575 flags.go:64] FLAG: --cluster-dns="[]" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543604 2575 flags.go:64] FLAG: --cluster-domain="" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543608 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543611 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543615 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543618 2575 flags.go:64] FLAG: --container-log-max-files="5" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543622 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 06 19:27:37.545660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543625 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543628 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543631 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543634 2575 flags.go:64] FLAG: --contention-profiling="false" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543637 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543640 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543643 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543646 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543650 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543653 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543656 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543659 2575 flags.go:64] FLAG: --enable-load-reader="false" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543661 2575 flags.go:64] FLAG: --enable-server="true" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543664 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543669 2575 flags.go:64] FLAG: --event-burst="100" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543672 2575 flags.go:64] FLAG: --event-qps="50" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543675 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543680 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543683 2575 flags.go:64] FLAG: --eviction-hard="" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543687 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543690 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543692 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543695 2575 flags.go:64] FLAG: --eviction-soft="" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543698 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543701 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 06 19:27:37.546247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543704 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543707 2575 flags.go:64] FLAG: --experimental-mounter-path="" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543710 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543713 2575 flags.go:64] FLAG: --fail-swap-on="true" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543716 2575 flags.go:64] FLAG: --feature-gates="" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543720 2575 flags.go:64] FLAG: --file-check-frequency="20s" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543723 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543726 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543729 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543732 2575 flags.go:64] FLAG: --healthz-port="10248" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543735 2575 flags.go:64] FLAG: --help="false" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543737 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543740 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543743 2575 flags.go:64] FLAG: --http-check-frequency="20s" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543761 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543764 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543767 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543770 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543773 2575 flags.go:64] FLAG: --image-service-endpoint="" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543776 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543779 2575 flags.go:64] FLAG: --kube-api-burst="100" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543781 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543785 2575 flags.go:64] FLAG: --kube-api-qps="50" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543788 2575 flags.go:64] FLAG: --kube-reserved="" Sep 06 19:27:37.546950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543792 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543794 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543797 2575 flags.go:64] FLAG: --kubelet-cgroups="" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543800 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543803 2575 flags.go:64] FLAG: --lock-file="" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543806 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543809 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543812 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543817 2575 flags.go:64] FLAG: --log-json-split-stream="false" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543820 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543823 2575 flags.go:64] FLAG: --log-text-split-stream="false" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543826 2575 flags.go:64] FLAG: --logging-format="text" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543829 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543833 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543836 2575 flags.go:64] FLAG: --manifest-url="" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543839 2575 flags.go:64] FLAG: --manifest-url-header="" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543843 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543847 2575 flags.go:64] FLAG: --max-open-files="1000000" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543851 2575 flags.go:64] FLAG: --max-pods="110" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543854 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543856 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543860 2575 flags.go:64] FLAG: --memory-manager-policy="None" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543863 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543866 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543868 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 06 19:27:37.547531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543871 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543879 2575 flags.go:64] FLAG: --node-status-max-images="50" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543882 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543885 2575 flags.go:64] FLAG: --oom-score-adj="-999" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543888 2575 flags.go:64] FLAG: --pod-cidr="" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543890 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543895 2575 flags.go:64] FLAG: --pod-manifest-path="" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543898 2575 flags.go:64] FLAG: --pod-max-pids="-1" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543905 2575 flags.go:64] FLAG: --pods-per-core="0" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543908 2575 flags.go:64] FLAG: --port="10250" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543911 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543914 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-016e0e778927ccaea" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543917 2575 flags.go:64] FLAG: --qos-reserved="" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543920 2575 flags.go:64] FLAG: --read-only-port="10255" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543923 2575 flags.go:64] FLAG: --register-node="true" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543925 2575 flags.go:64] FLAG: --register-with-taints="" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543929 2575 flags.go:64] FLAG: --registry-burst="10" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543932 2575 flags.go:64] FLAG: --registry-qps="5" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543935 2575 flags.go:64] FLAG: --reserved-cpus="" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543938 2575 flags.go:64] FLAG: --reserved-memory="" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543942 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543945 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543948 2575 flags.go:64] FLAG: --rotate-certificates="false" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543951 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543953 2575 flags.go:64] FLAG: --runonce="false" Sep 06 19:27:37.548187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543956 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543959 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543962 2575 flags.go:64] FLAG: --seccomp-default="false" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543965 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543968 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543971 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543974 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543977 2575 flags.go:64] FLAG: --storage-driver-password="root" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543980 2575 flags.go:64] FLAG: --storage-driver-secure="false" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543983 2575 flags.go:64] FLAG: --storage-driver-table="stats" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543985 2575 flags.go:64] FLAG: --storage-driver-user="root" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543988 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543991 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543994 2575 flags.go:64] FLAG: --system-cgroups="" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.543997 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544005 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544008 2575 flags.go:64] FLAG: --tls-cert-file="" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544010 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544018 2575 flags.go:64] FLAG: --tls-min-version="" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544021 2575 flags.go:64] FLAG: --tls-private-key-file="" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544024 2575 flags.go:64] FLAG: --topology-manager-policy="none" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544027 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544029 2575 flags.go:64] FLAG: --topology-manager-scope="container" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544032 2575 flags.go:64] FLAG: --v="2" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544037 2575 flags.go:64] FLAG: --version="false" Sep 06 19:27:37.548828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544042 2575 flags.go:64] FLAG: --vmodule="" Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544049 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.544052 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544160 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544165 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544168 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544171 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544174 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544177 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544180 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544183 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544186 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544189 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544192 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544194 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544197 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544200 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544202 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544205 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544208 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 19:27:37.549443 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544210 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544213 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544216 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544219 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544221 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544224 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544227 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544229 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544232 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544235 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544237 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544241 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544243 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544246 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544249 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544252 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544254 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544257 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544259 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544262 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 19:27:37.549955 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544264 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544267 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544269 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544272 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544275 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544277 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544280 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544282 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544285 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544287 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544290 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544294 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544298 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544301 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544304 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544307 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544310 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544313 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544316 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544319 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 19:27:37.550452 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544322 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544325 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544327 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544332 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544335 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544337 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544340 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544343 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544346 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544348 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544351 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544353 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544356 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544358 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544361 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544363 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544366 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544369 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544371 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544374 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 19:27:37.551014 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544376 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544379 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544381 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544384 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544387 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544389 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544392 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544395 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544398 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544402 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544406 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544409 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544413 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544415 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544418 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544422 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544424 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544427 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544429 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 19:27:37.551544 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544432 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 19:27:37.552073 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544435 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 19:27:37.552073 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.544438 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 19:27:37.552073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.545113 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 19:27:37.552875 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.552857 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 06 19:27:37.552875 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.552875 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 06 19:27:37.553616 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.553607 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 06 19:27:37.553647 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.553617 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 06 19:27:37.553693 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553684 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 19:27:37.553693 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553693 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553696 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553700 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553703 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553707 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553709 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553713 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553717 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553720 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553723 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553726 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553729 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553731 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553734 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553737 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553739 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553743 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553745 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553765 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 19:27:37.553763 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553768 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553771 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553774 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553777 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553779 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553783 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553786 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553788 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553791 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553793 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553796 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553799 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553801 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553804 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553806 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553809 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553812 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553814 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553819 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 19:27:37.554230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553822 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553825 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553828 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553831 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553834 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553836 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553839 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553841 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553844 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553846 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553849 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553851 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553854 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553856 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553859 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553865 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553868 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553871 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553874 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553877 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 19:27:37.554702 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553880 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553883 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553886 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553889 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553891 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553894 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553897 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553899 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553902 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553904 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553907 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553909 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553912 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553915 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553917 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553920 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553922 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553925 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553927 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553930 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 19:27:37.555210 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553932 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553935 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553937 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553940 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553943 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553945 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553948 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553951 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553953 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553956 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553958 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553961 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553964 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553967 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553969 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553972 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553974 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553977 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553979 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 19:27:37.555676 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.553982 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.553986 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554116 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554120 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554124 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554127 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554129 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554132 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554135 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554137 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554140 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554143 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554145 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554147 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554150 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 19:27:37.556155 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554152 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554155 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554157 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554160 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554163 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554166 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554169 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554172 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554174 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554177 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554179 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554182 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554185 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554187 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554190 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554192 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554195 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554197 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554200 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554202 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 19:27:37.556519 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554205 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554207 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554210 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554213 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554215 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554218 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554220 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554223 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554225 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554228 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554230 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554233 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554236 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554238 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554241 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554244 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554246 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554249 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554252 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 19:27:37.557031 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554255 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554257 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554260 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554262 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554266 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554270 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554273 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554276 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554279 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554281 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554285 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554296 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554299 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554302 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554305 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554308 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554310 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554313 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554316 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 19:27:37.557506 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554318 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554322 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554325 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554327 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554330 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554333 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554335 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554337 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554340 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554342 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554345 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554348 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554351 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554354 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554357 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554360 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554363 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554365 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554368 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 19:27:37.557987 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554371 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554374 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554376 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554379 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554381 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554384 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554386 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554389 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:37.554391 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.554396 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.554495 2575 server.go:962] "Client rotation is on, will bootstrap in background" Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.556859 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.557642 2575 server.go:1019] "Starting client certificate rotation" Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.557740 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 06 19:27:37.558470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.557794 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 06 19:27:37.560319 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.560307 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 06 19:27:37.561653 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.561634 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 06 19:27:37.573794 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.573771 2575 log.go:25] "Validated CRI v1 runtime API" Sep 06 19:27:37.579423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.579402 2575 log.go:25] "Validated CRI v1 image API" Sep 06 19:27:37.580344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.580327 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 06 19:27:37.582476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.582457 2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c0b4c327-4717-4e88-8765-358f0d2d6943:/dev/nvme0n1p4 e70b66b8-5a82-4810-903b-5d3c486da744:/dev/nvme0n1p3] Sep 06 19:27:37.582558 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.582475 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 06 19:27:37.584374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.584355 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 06 19:27:37.587723 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.587608 2575 manager.go:216] Machine: {Timestamp:2026-09-06 19:27:37.586362498 +0000 UTC m=+0.320967222 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100475 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2882159621eda4e933ddd111efd4a9 SystemUUID:ec288215-9621-eda4-e933-ddd111efd4a9 BootID:902ebd13-9e49-4918-ac3c-23847759a671 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:de:72:83:f0:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:de:72:83:f0:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:81:7e:43:5e:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 06 19:27:37.587723 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.587719 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 06 19:27:37.587863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.587816 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 06 19:27:37.591065 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.591042 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 06 19:27:37.591201 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.591068 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 06 19:27:37.591919 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.591909 2575 topology_manager.go:138] "Creating topology manager with none policy" Sep 06 19:27:37.591949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.591921 2575 container_manager_linux.go:309] "Creating device plugin manager" Sep 06 19:27:37.591949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.591929 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 06 19:27:37.592780 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.592769 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 06 19:27:37.594629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.594619 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 06 19:27:37.598330 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598317 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 06 19:27:37.598447 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598438 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 06 19:27:37.598478 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598470 2575 kubelet.go:486] "Attempting to sync node with API server" Sep 06 19:27:37.598505 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598484 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 06 19:27:37.598551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598509 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 06 19:27:37.598551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598523 2575 kubelet.go:398] "Adding apiserver pod source" Sep 06 19:27:37.598551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.598535 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 06 19:27:37.599081 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.599069 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 06 19:27:37.599130 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.599088 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 06 19:27:37.601558 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.601538 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 06 19:27:37.604346 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.604332 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 06 19:27:37.604408 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.604363 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 06 19:27:37.605194 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605179 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 06 19:27:37.605194 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605197 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605203 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605209 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605217 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605222 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605237 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605242 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605248 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605258 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605267 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 06 19:27:37.605287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.605272 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 06 19:27:37.606628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.606615 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 06 19:27:37.606628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.606627 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 06 19:27:37.608564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.608545 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-znx7m" Sep 06 19:27:37.609029 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.609001 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 06 19:27:37.609075 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.609046 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 06 19:27:37.609157 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.609124 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 06 19:27:37.610623 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.610465 2575 server.go:1268] "Started kubelet" Sep 06 19:27:37.610719 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.610546 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 06 19:27:37.610797 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.610550 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 06 19:27:37.610861 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.610846 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 06 19:27:37.611473 ip-10-0-135-236 systemd[1]: Started Kubernetes Kubelet. Sep 06 19:27:37.612344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.612328 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 06 19:27:37.613019 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.613005 2575 server.go:310] "Adding debug handlers to kubelet server" Sep 06 19:27:37.615345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.615323 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-znx7m" Sep 06 19:27:37.616154 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.616132 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 06 19:27:37.619303 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.619285 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 06 19:27:37.619303 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.619305 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 06 19:27:37.619493 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.619373 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:37.620053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.619971 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 06 19:27:37.620053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.619989 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 06 19:27:37.620053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620028 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 06 19:27:37.620234 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620148 2575 reconstruct.go:97] "Volume reconstruction finished" Sep 06 19:27:37.620234 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620159 2575 reconciler.go:29] "Reconciler: start to sync state" Sep 06 19:27:37.620593 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620576 2575 factory.go:153] Registering CRI-O factory Sep 06 19:27:37.620593 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620595 2575 factory.go:223] Registration of the crio container factory successfully Sep 06 19:27:37.620744 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620642 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 06 19:27:37.620744 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620652 2575 factory.go:55] Registering systemd factory Sep 06 19:27:37.620744 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620660 2575 factory.go:223] Registration of the systemd container factory successfully Sep 06 19:27:37.620744 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620675 2575 factory.go:103] Registering Raw factory Sep 06 19:27:37.620744 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.620688 2575 manager.go:1196] Started watching for new ooms in manager Sep 06 19:27:37.621017 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.621006 2575 manager.go:349] Starting recovery of all containers Sep 06 19:27:37.622298 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.622273 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:37.625219 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.625196 2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-236.ec2.internal\" not found" node="ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.630441 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.630422 2575 manager.go:354] Recovery completed Sep 06 19:27:37.631629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.631609 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 06 19:27:37.636046 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.636032 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.637695 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.637676 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.637823 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.637709 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.637823 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.637725 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.638285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.638271 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 06 19:27:37.638285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.638283 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.638398 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.638283 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 06 19:27:37.638462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.638398 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 06 19:27:37.638996 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.638978 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.639091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.639009 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.639091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.639025 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.639634 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.639623 2575 policy_none.go:49] "None policy: Start" Sep 06 19:27:37.639667 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.639639 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 06 19:27:37.639667 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.639648 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 06 19:27:37.640736 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.640726 2575 policy_none.go:47] "Start" Sep 06 19:27:37.671417 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.671398 2575 manager.go:337] "Starting Device Plugin manager" Sep 06 19:27:37.671512 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.671434 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 06 19:27:37.671512 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.671445 2575 server.go:87] "Starting device plugin registration server" Sep 06 19:27:37.671795 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.671782 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 06 19:27:37.671864 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.671798 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 06 19:27:37.671970 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.671923 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 06 19:27:37.672024 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.672006 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 06 19:27:37.672024 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.672017 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 06 19:27:37.672598 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.672572 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 06 19:27:37.672697 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.672633 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:37.755739 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.755702 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 06 19:27:37.755906 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.755776 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 06 19:27:37.755906 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.755810 2575 kubelet.go:2448] "Starting kubelet main sync loop" Sep 06 19:27:37.755906 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.755851 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 06 19:27:37.757686 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.757665 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:37.772875 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.772853 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.773704 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.773686 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.773831 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.773725 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.773831 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.773737 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.773831 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.773783 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.781614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.781600 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.781701 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.781622 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-236.ec2.internal\": node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:37.796257 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.796239 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:37.856922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.856885 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal"] Sep 06 19:27:37.857005 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.856991 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.858740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.858724 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.858837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.858772 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.858837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.858789 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.860352 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.860339 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.860495 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.860480 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.860529 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.860509 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.861204 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.861188 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.861305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.861218 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.861305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.861228 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.861305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.861188 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.861305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.861294 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.861305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.861307 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.862521 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.862508 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.862577 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.862534 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 19:27:37.863273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.863256 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientMemory" Sep 06 19:27:37.863374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.863288 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 19:27:37.863374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.863302 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeHasSufficientPID" Sep 06 19:27:37.886269 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.886253 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-236.ec2.internal\" not found" node="ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.890542 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.890526 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-236.ec2.internal\" not found" node="ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.896726 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.896712 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:37.921864 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.921846 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a7a76a58af3f5a735a46f27cbfd95e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal\" (UID: \"7a7a76a58af3f5a735a46f27cbfd95e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.921967 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.921876 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a7a76a58af3f5a735a46f27cbfd95e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal\" (UID: \"7a7a76a58af3f5a735a46f27cbfd95e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.921967 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:37.921902 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72e1c97d5f33b9f41328e5f5e35a0ce0-config\") pod \"kube-apiserver-proxy-ip-10-0-135-236.ec2.internal\" (UID: \"72e1c97d5f33b9f41328e5f5e35a0ce0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" Sep 06 19:27:37.997041 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:37.997018 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.022420 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.022393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a7a76a58af3f5a735a46f27cbfd95e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal\" (UID: \"7a7a76a58af3f5a735a46f27cbfd95e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.022520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.022432 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a7a76a58af3f5a735a46f27cbfd95e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal\" (UID: \"7a7a76a58af3f5a735a46f27cbfd95e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.022520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.022434 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a7a76a58af3f5a735a46f27cbfd95e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal\" (UID: \"7a7a76a58af3f5a735a46f27cbfd95e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.022520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.022492 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72e1c97d5f33b9f41328e5f5e35a0ce0-config\") pod \"kube-apiserver-proxy-ip-10-0-135-236.ec2.internal\" (UID: \"72e1c97d5f33b9f41328e5f5e35a0ce0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.022633 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.022474 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a7a76a58af3f5a735a46f27cbfd95e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal\" (UID: \"7a7a76a58af3f5a735a46f27cbfd95e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.022633 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.022525 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72e1c97d5f33b9f41328e5f5e35a0ce0-config\") pod \"kube-apiserver-proxy-ip-10-0-135-236.ec2.internal\" (UID: \"72e1c97d5f33b9f41328e5f5e35a0ce0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.097710 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.097645 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.190332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.190307 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.195072 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.194492 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.198767 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.198735 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.299267 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.299226 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.399769 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.399692 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.500279 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.500245 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.520918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.520895 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:38.557765 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.557734 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 06 19:27:38.558203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.557859 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 06 19:27:38.558203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.557900 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 06 19:27:38.558203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.557912 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 06 19:27:38.601208 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.601174 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.620697 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.620650 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 19:22:38 +0000 UTC" deadline="2028-05-15 08:44:39.621698762 +0000 UTC" Sep 06 19:27:38.620697 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.620688 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14797h17m1.001014211s" Sep 06 19:27:38.701866 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:38.701788 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-236.ec2.internal\" not found" Sep 06 19:27:38.712697 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.712671 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:38.720603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.720583 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.729577 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.729555 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 06 19:27:38.730495 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.730473 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" Sep 06 19:27:38.739503 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.739484 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 06 19:27:38.884214 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:38.884176 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a7a76a58af3f5a735a46f27cbfd95e9.slice/crio-0297d87202cd1aff13ecd375c83ca83a09b49e3bf86f6862379213e7352b8eef WatchSource:0}: Error finding container 0297d87202cd1aff13ecd375c83ca83a09b49e3bf86f6862379213e7352b8eef: Status 404 returned error can't find the container with id 0297d87202cd1aff13ecd375c83ca83a09b49e3bf86f6862379213e7352b8eef Sep 06 19:27:38.884585 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:38.884569 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e1c97d5f33b9f41328e5f5e35a0ce0.slice/crio-22cbcba21c2dfbc8bbef5facbc4e3be8f07e6b66b8fd60f94e3a23cf228c4304 WatchSource:0}: Error finding container 22cbcba21c2dfbc8bbef5facbc4e3be8f07e6b66b8fd60f94e3a23cf228c4304: Status 404 returned error can't find the container with id 22cbcba21c2dfbc8bbef5facbc4e3be8f07e6b66b8fd60f94e3a23cf228c4304 Sep 06 19:27:38.889536 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:38.889078 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 19:27:39.599635 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.599547 2575 apiserver.go:52] "Watching apiserver" Sep 06 19:27:39.609327 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.609285 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 06 19:27:39.610602 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.610561 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc","openshift-cluster-node-tuning-operator/tuned-pfljw","openshift-dns/node-resolver-jbbtr","openshift-image-registry/node-ca-th522","openshift-multus/multus-additional-cni-plugins-hmmz8","openshift-network-diagnostics/network-check-target-9j4ml","openshift-network-operator/iptables-alerter-mzghj","kube-system/global-pull-secret-syncer-4f985","kube-system/konnectivity-agent-qnljk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal","openshift-multus/multus-v7nd9","openshift-multus/network-metrics-daemon-p2qnt","openshift-ovn-kubernetes/ovnkube-node-zqg62","kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal"] Sep 06 19:27:39.613598 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.613575 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.615082 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.615057 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.616415 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.616388 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 06 19:27:39.616521 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.616458 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q8fg6\"" Sep 06 19:27:39.616717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.616701 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.616806 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.616718 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.617315 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.618233 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.618282 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.618448 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.618464 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.618907 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.618941 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kj5rk\"" Sep 06 19:27:39.619235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.619075 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 06 19:27:39.619693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.619395 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.619693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.619431 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.619693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.619586 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 06 19:27:39.620074 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.620049 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.620173 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.620152 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w9q45\"" Sep 06 19:27:39.621341 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.620917 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.621950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.621927 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.622715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.622324 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 06 19:27:39.622715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.622341 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.622715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.622577 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k54l6\"" Sep 06 19:27:39.623336 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.623051 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:39.623336 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.623128 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:39.624006 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.623680 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.624006 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.623737 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.624006 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.623777 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 06 19:27:39.624006 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.623788 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ssxfn\"" Sep 06 19:27:39.624457 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.624436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.624457 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.624454 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 06 19:27:39.624595 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.624533 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 06 19:27:39.626450 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.626429 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.627275 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.627253 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 06 19:27:39.627395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.627297 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 06 19:27:39.627395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.627309 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pfx8g\"" Sep 06 19:27:39.628672 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.628510 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.629218 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.629197 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 06 19:27:39.629375 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.629358 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.629453 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.629407 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.629902 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.629882 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bzjtn\"" Sep 06 19:27:39.631030 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.631008 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 06 19:27:39.631294 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.631275 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c7jfw\"" Sep 06 19:27:39.632419 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.632399 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 06 19:27:39.633492 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633467 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.633592 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.633537 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633806 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5zfr\" (UniqueName: \"kubernetes.io/projected/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-kube-api-access-v5zfr\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79361bdc-daa9-4602-8bdb-e9d0c029546f-host-slash\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633889 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-slash\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633904 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-cni-bin\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-cni-netd\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7w59\" (UniqueName: \"kubernetes.io/projected/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-kube-api-access-k7w59\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.633982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.634017 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-kubelet\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.634049 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-var-lib-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.634064 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-log-socket\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.634079 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-env-overrides\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.634170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.634095 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/052d0c66-9223-419b-980f-e592d8e633f0-hosts-file\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.636746 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.635336 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 06 19:27:39.636746 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.635950 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 06 19:27:39.636746 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636318 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:39.636746 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.636405 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:39.636746 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636628 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d0c66-9223-419b-980f-e592d8e633f0-tmp-dir\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.636746 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-system-cni-dir\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636740 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-run-netns\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636793 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-systemd\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636824 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cnibin\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636914 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-systemd-units\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.636944 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-etc-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637044 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/79361bdc-daa9-4602-8bdb-e9d0c029546f-iptables-alerter-script\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.637109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637077 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-ovn\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637112 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-node-log\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637158 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovnkube-script-lib\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-host\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.637488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637255 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thrsx\" (UniqueName: \"kubernetes.io/projected/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-kube-api-access-thrsx\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.637488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637314 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637514 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637579 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637609 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovnkube-config\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637640 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovn-node-metrics-cert\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5dc\" (UniqueName: \"kubernetes.io/projected/052d0c66-9223-419b-980f-e592d8e633f0-kube-api-access-sq5dc\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637724 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-serviceca\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.637939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwzbx\" (UniqueName: \"kubernetes.io/projected/79361bdc-daa9-4602-8bdb-e9d0c029546f-kube-api-access-rwzbx\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.638276 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.637962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.640423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.639501 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fm62m\"" Sep 06 19:27:39.654232 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.654191 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6wflr" Sep 06 19:27:39.661802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.661777 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6wflr" Sep 06 19:27:39.721279 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.721250 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 06 19:27:39.738422 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-multus-certs\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.738552 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738437 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-host\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.738552 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738462 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47b1c86e-16a0-4f33-8436-1b186e573396-tmp\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.738552 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738514 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxkxl\" (UniqueName: \"kubernetes.io/projected/47b1c86e-16a0-4f33-8436-1b186e573396-kube-api-access-bxkxl\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.738727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738568 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae7b9294-acfa-4b8d-88af-91860510096d-konnectivity-ca\") pod \"konnectivity-agent-qnljk\" (UID: \"ae7b9294-acfa-4b8d-88af-91860510096d\") " pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.738727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-conf-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.738727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738646 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-cni-bin\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.738727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738672 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.738727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738699 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae7b9294-acfa-4b8d-88af-91860510096d-agent-certs\") pod \"konnectivity-agent-qnljk\" (UID: \"ae7b9294-acfa-4b8d-88af-91860510096d\") " pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.738727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738723 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-hostroot\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738798 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738829 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-kubelet\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-var-lib-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-log-socket\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-env-overrides\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.738933 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/349c8685-ef1a-4e22-b5d4-20fc72aac46a-cni-binary-copy\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739021 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/052d0c66-9223-419b-980f-e592d8e633f0-hosts-file\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.739054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739050 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-system-cni-dir\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-run-netns\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739106 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-cni-bin\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739143 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-etc-kubernetes\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-socket-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739225 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd7bt\" (UniqueName: \"kubernetes.io/projected/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-kube-api-access-xd7bt\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739256 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-kubelet-config\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739280 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zp2b\" (UniqueName: \"kubernetes.io/projected/349c8685-ef1a-4e22-b5d4-20fc72aac46a-kube-api-access-6zp2b\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739304 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47b1c86e-16a0-4f33-8436-1b186e573396-etc-tuned\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-systemd-units\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-modprobe-d\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.739458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-systemd\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739475 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-log-socket\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-ovn\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739530 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-cni-bin\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739535 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovnkube-script-lib\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-dbus\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-system-cni-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739662 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysconfig\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-sys\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739748 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739780 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovnkube-config\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739805 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739870 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-kubelet\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739452 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-systemd-units\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.740132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.739985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysctl-conf\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740016 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwzbx\" (UniqueName: \"kubernetes.io/projected/79361bdc-daa9-4602-8bdb-e9d0c029546f-kube-api-access-rwzbx\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740075 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-k8s-cni-cncf-io\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-kubelet\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740136 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5zfr\" (UniqueName: \"kubernetes.io/projected/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-kube-api-access-v5zfr\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740165 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79361bdc-daa9-4602-8bdb-e9d0c029546f-host-slash\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740191 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-slash\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740235 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-cni-netd\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740265 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7w59\" (UniqueName: \"kubernetes.io/projected/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-kube-api-access-k7w59\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740292 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-etc-selinux\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740320 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-sys-fs\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740347 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-socket-dir-parent\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-run\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d0c66-9223-419b-980f-e592d8e633f0-tmp-dir\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740434 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-systemd\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-cnibin\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740517 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-cni-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-registration-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-device-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cnibin\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740631 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-etc-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740676 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-netns\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-cni-multus\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740921 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/79361bdc-daa9-4602-8bdb-e9d0c029546f-iptables-alerter-script\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.740970 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-node-log\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741004 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-daemon-config\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741034 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-lib-modules\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741064 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-var-lib-kubelet\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741101 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-host\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741140 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thrsx\" (UniqueName: \"kubernetes.io/projected/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-kube-api-access-thrsx\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741240 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovn-node-metrics-cert\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.741910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741324 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4bdv\" (UniqueName: \"kubernetes.io/projected/46414df0-60db-46f7-91d2-5378dff237aa-kube-api-access-x4bdv\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741352 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-kubernetes\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysctl-d\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741382 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-env-overrides\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741409 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5dc\" (UniqueName: \"kubernetes.io/projected/052d0c66-9223-419b-980f-e592d8e633f0-kube-api-access-sq5dc\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-serviceca\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741479 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/052d0c66-9223-419b-980f-e592d8e633f0-hosts-file\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741533 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-run-netns\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741564 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-system-cni-dir\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741618 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-ovn\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741654 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-var-lib-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.741694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742236 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovnkube-script-lib\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742277 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742336 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-etc-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742609 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-serviceca\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.742868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742709 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742907 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/79361bdc-daa9-4602-8bdb-e9d0c029546f-iptables-alerter-script\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.742969 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-node-log\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743009 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052d0c66-9223-419b-980f-e592d8e633f0-tmp-dir\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743024 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-host\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743084 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-systemd\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743130 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743173 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-cnibin\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743443 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-run-openvswitch\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.743690 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743549 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovnkube-config\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.744083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743721 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.744083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743731 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79361bdc-daa9-4602-8bdb-e9d0c029546f-host-slash\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.744083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743866 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.744083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743903 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-slash\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.744083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.743926 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-host-cni-netd\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.744410 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.744215 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 06 19:27:39.748890 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.748866 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:39.749806 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.749751 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-ovn-node-metrics-cert\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.754052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.753995 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5dc\" (UniqueName: \"kubernetes.io/projected/052d0c66-9223-419b-980f-e592d8e633f0-kube-api-access-sq5dc\") pod \"node-resolver-jbbtr\" (UID: \"052d0c66-9223-419b-980f-e592d8e633f0\") " pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.759684 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.756659 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thrsx\" (UniqueName: \"kubernetes.io/projected/24a08ec0-deb6-41e9-93ff-8234c06fd1c4-kube-api-access-thrsx\") pod \"node-ca-th522\" (UID: \"24a08ec0-deb6-41e9-93ff-8234c06fd1c4\") " pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.761304 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.761252 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwzbx\" (UniqueName: \"kubernetes.io/projected/79361bdc-daa9-4602-8bdb-e9d0c029546f-kube-api-access-rwzbx\") pod \"iptables-alerter-mzghj\" (UID: \"79361bdc-daa9-4602-8bdb-e9d0c029546f\") " pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.761411 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.761356 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5zfr\" (UniqueName: \"kubernetes.io/projected/16b7f01c-8c0b-4b1d-944e-9ca9418371b3-kube-api-access-v5zfr\") pod \"multus-additional-cni-plugins-hmmz8\" (UID: \"16b7f01c-8c0b-4b1d-944e-9ca9418371b3\") " pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.767717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.767689 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7w59\" (UniqueName: \"kubernetes.io/projected/4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3-kube-api-access-k7w59\") pod \"ovnkube-node-zqg62\" (UID: \"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.768789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.768720 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" event={"ID":"7a7a76a58af3f5a735a46f27cbfd95e9","Type":"ContainerStarted","Data":"0297d87202cd1aff13ecd375c83ca83a09b49e3bf86f6862379213e7352b8eef"} Sep 06 19:27:39.770782 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.770729 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" event={"ID":"72e1c97d5f33b9f41328e5f5e35a0ce0","Type":"ContainerStarted","Data":"22cbcba21c2dfbc8bbef5facbc4e3be8f07e6b66b8fd60f94e3a23cf228c4304"} Sep 06 19:27:39.784040 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.784017 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:39.842264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842230 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-system-cni-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysconfig\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-sys\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842415 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysconfig\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.842440 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-system-cni-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:39.842504 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.842507 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:27:40.342483751 +0000 UTC m=+3.077088491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842508 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-sys\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysctl-conf\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-k8s-cni-cncf-io\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-kubelet\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-etc-selinux\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-sys-fs\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-socket-dir-parent\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842667 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-k8s-cni-cncf-io\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842682 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-run\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842709 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-cnibin\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842734 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-cni-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842760 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-registration-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842780 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-sys-fs\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-device-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842837 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysctl-conf\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-netns\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842856 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-kubelet\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.842940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842880 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-cni-multus\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842906 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-daemon-config\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842926 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-lib-modules\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842946 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-var-lib-kubelet\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842947 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-etc-selinux\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4bdv\" (UniqueName: \"kubernetes.io/projected/46414df0-60db-46f7-91d2-5378dff237aa-kube-api-access-x4bdv\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.842997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-netns\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843000 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-kubernetes\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843000 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-cni-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysctl-d\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843047 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-socket-dir-parent\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-multus-certs\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843063 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-registration-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-host\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843108 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47b1c86e-16a0-4f33-8436-1b186e573396-tmp\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843118 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-device-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843149 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-cnibin\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxkxl\" (UniqueName: \"kubernetes.io/projected/47b1c86e-16a0-4f33-8436-1b186e573396-kube-api-access-bxkxl\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.843896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843182 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-kubernetes\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae7b9294-acfa-4b8d-88af-91860510096d-konnectivity-ca\") pod \"konnectivity-agent-qnljk\" (UID: \"ae7b9294-acfa-4b8d-88af-91860510096d\") " pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843207 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-cni-multus\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843213 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-conf-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843220 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-var-lib-kubelet\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843254 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae7b9294-acfa-4b8d-88af-91860510096d-agent-certs\") pod \"konnectivity-agent-qnljk\" (UID: \"ae7b9294-acfa-4b8d-88af-91860510096d\") " pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-hostroot\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843349 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/349c8685-ef1a-4e22-b5d4-20fc72aac46a-cni-binary-copy\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-cni-bin\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-etc-kubernetes\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843463 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-socket-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd7bt\" (UniqueName: \"kubernetes.io/projected/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-kube-api-access-xd7bt\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843498 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-lib-modules\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843518 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-kubelet-config\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zp2b\" (UniqueName: \"kubernetes.io/projected/349c8685-ef1a-4e22-b5d4-20fc72aac46a-kube-api-access-6zp2b\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.844918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843568 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-run-multus-certs\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843567 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47b1c86e-16a0-4f33-8436-1b186e573396-etc-tuned\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843616 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-run\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-modprobe-d\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-systemd\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843689 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-dbus\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843711 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-modprobe-d\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843795 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-dbus\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843803 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-daemon-config\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843864 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-host\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843884 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-etc-kubernetes\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.843935 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-multus-conf-dir\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844129 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-systemd\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844186 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-kubelet-config\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844230 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-hostroot\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844237 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae7b9294-acfa-4b8d-88af-91860510096d-konnectivity-ca\") pod \"konnectivity-agent-qnljk\" (UID: \"ae7b9294-acfa-4b8d-88af-91860510096d\") " pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844239 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844347 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47b1c86e-16a0-4f33-8436-1b186e573396-etc-sysctl-d\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.845669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844350 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-socket-dir\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.846563 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.844357 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:39.846563 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.844416 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:27:40.344399116 +0000 UTC m=+3.079003835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:39.846563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.844874 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/349c8685-ef1a-4e22-b5d4-20fc72aac46a-host-var-lib-cni-bin\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.846563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.845185 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/349c8685-ef1a-4e22-b5d4-20fc72aac46a-cni-binary-copy\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.846934 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.846908 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47b1c86e-16a0-4f33-8436-1b186e573396-tmp\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.847646 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.847617 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae7b9294-acfa-4b8d-88af-91860510096d-agent-certs\") pod \"konnectivity-agent-qnljk\" (UID: \"ae7b9294-acfa-4b8d-88af-91860510096d\") " pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.848262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.848242 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47b1c86e-16a0-4f33-8436-1b186e573396-etc-tuned\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.851710 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.851657 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:27:39.851710 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.851688 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:27:39.851710 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.851704 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:39.851909 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:39.851784 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:27:40.351766774 +0000 UTC m=+3.086371503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:39.855922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.855897 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd7bt\" (UniqueName: \"kubernetes.io/projected/8bf1148c-8d55-47ee-b7cb-2ac95c6fc605-kube-api-access-xd7bt\") pod \"aws-ebs-csi-driver-node-8glqc\" (UID: \"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.856015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.855973 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4bdv\" (UniqueName: \"kubernetes.io/projected/46414df0-60db-46f7-91d2-5378dff237aa-kube-api-access-x4bdv\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:39.856214 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.856191 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zp2b\" (UniqueName: \"kubernetes.io/projected/349c8685-ef1a-4e22-b5d4-20fc72aac46a-kube-api-access-6zp2b\") pod \"multus-v7nd9\" (UID: \"349c8685-ef1a-4e22-b5d4-20fc72aac46a\") " pod="openshift-multus/multus-v7nd9" Sep 06 19:27:39.856768 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.856707 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxkxl\" (UniqueName: \"kubernetes.io/projected/47b1c86e-16a0-4f33-8436-1b186e573396-kube-api-access-bxkxl\") pod \"tuned-pfljw\" (UID: \"47b1c86e-16a0-4f33-8436-1b186e573396\") " pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:39.932762 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.932724 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzghj" Sep 06 19:27:39.944030 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.944003 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:27:39.957077 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.957053 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jbbtr" Sep 06 19:27:39.960972 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.960948 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-th522" Sep 06 19:27:39.970806 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.970779 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" Sep 06 19:27:39.978922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.978886 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:27:39.988384 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.988366 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" Sep 06 19:27:39.996894 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:39.996871 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pfljw" Sep 06 19:27:40.003965 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.003943 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v7nd9" Sep 06 19:27:40.348295 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.348194 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:40.348295 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.348271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:40.348528 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.348371 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:40.348528 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.348406 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:40.348528 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.348457 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:27:41.348430989 +0000 UTC m=+4.083035714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:40.348528 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.348475 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:27:41.348467437 +0000 UTC m=+4.083072148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:40.449767 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.449725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:40.449943 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.449870 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:27:40.449943 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.449890 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:27:40.449943 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.449902 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:40.450079 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.449964 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:27:41.449946767 +0000 UTC m=+4.184551505 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:40.665170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.665083 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 19:22:40 +0000 UTC" deadline="2028-02-02 22:12:00.479493184 +0000 UTC" Sep 06 19:27:40.665170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.665122 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12338h44m19.814376041s" Sep 06 19:27:40.736056 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.736021 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 19:27:40.756823 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.756782 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:40.756979 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:40.756825 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:40.756979 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.756914 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:40.757088 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:40.757066 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:40.974065 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:40.974039 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a08ec0_deb6_41e9_93ff_8234c06fd1c4.slice/crio-03892c0bf9442d3c2384cb28200f79198142319c54b8609dd8b331df32fb34b6 WatchSource:0}: Error finding container 03892c0bf9442d3c2384cb28200f79198142319c54b8609dd8b331df32fb34b6: Status 404 returned error can't find the container with id 03892c0bf9442d3c2384cb28200f79198142319c54b8609dd8b331df32fb34b6 Sep 06 19:27:40.974743 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:40.974721 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b1c86e_16a0_4f33_8436_1b186e573396.slice/crio-e3179d3324732efe9477d41b53fa62e1060409952d49ea28d92bd691c2a9ede8 WatchSource:0}: Error finding container e3179d3324732efe9477d41b53fa62e1060409952d49ea28d92bd691c2a9ede8: Status 404 returned error can't find the container with id e3179d3324732efe9477d41b53fa62e1060409952d49ea28d92bd691c2a9ede8 Sep 06 19:27:40.975868 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:40.975843 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf1148c_8d55_47ee_b7cb_2ac95c6fc605.slice/crio-ef7fbb5e3bf92377caf93a4065c4723cefc9a6516ae23cca3d2f5d6e13182e8a WatchSource:0}: Error finding container ef7fbb5e3bf92377caf93a4065c4723cefc9a6516ae23cca3d2f5d6e13182e8a: Status 404 returned error can't find the container with id ef7fbb5e3bf92377caf93a4065c4723cefc9a6516ae23cca3d2f5d6e13182e8a Sep 06 19:27:40.978098 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:40.978073 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79361bdc_daa9_4602_8bdb_e9d0c029546f.slice/crio-01ed91d549350d2d72305bd7119b172e3071ac1a0e638c5020e2e166bb395b18 WatchSource:0}: Error finding container 01ed91d549350d2d72305bd7119b172e3071ac1a0e638c5020e2e166bb395b18: Status 404 returned error can't find the container with id 01ed91d549350d2d72305bd7119b172e3071ac1a0e638c5020e2e166bb395b18 Sep 06 19:27:40.978976 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:40.978891 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052d0c66_9223_419b_980f_e592d8e633f0.slice/crio-cb3d94cbb2cbcc9c9734221ae8e97ecf01c10700fc69f981150fb5fab0e62127 WatchSource:0}: Error finding container cb3d94cbb2cbcc9c9734221ae8e97ecf01c10700fc69f981150fb5fab0e62127: Status 404 returned error can't find the container with id cb3d94cbb2cbcc9c9734221ae8e97ecf01c10700fc69f981150fb5fab0e62127 Sep 06 19:27:40.984474 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:27:40.984448 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349c8685_ef1a_4e22_b5d4_20fc72aac46a.slice/crio-2aed667fc6f986f53c01d13524042f84414b8e478fae712c979beb9f600f0d23 WatchSource:0}: Error finding container 2aed667fc6f986f53c01d13524042f84414b8e478fae712c979beb9f600f0d23: Status 404 returned error can't find the container with id 2aed667fc6f986f53c01d13524042f84414b8e478fae712c979beb9f600f0d23 Sep 06 19:27:41.356721 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.356687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:41.356913 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.356767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:41.356913 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.356892 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:41.356913 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.356901 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:41.357079 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.356956 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:27:43.35694039 +0000 UTC m=+6.091545102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:41.357079 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.356970 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:27:43.35696309 +0000 UTC m=+6.091567802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:41.457204 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.457158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:41.457482 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.457338 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:27:41.457482 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.457359 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:27:41.457482 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.457372 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:41.457482 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.457433 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:27:43.457412828 +0000 UTC m=+6.192017542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:41.666363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.666269 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 19:22:40 +0000 UTC" deadline="2028-02-10 06:21:05.15151461 +0000 UTC" Sep 06 19:27:41.666363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.666334 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12514h53m23.485185449s" Sep 06 19:27:41.757547 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.756960 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:41.757547 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:41.757110 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:41.792533 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.792048 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" event={"ID":"72e1c97d5f33b9f41328e5f5e35a0ce0","Type":"ContainerStarted","Data":"20fae60fbd25aabf055275623625e61a819675e67eaae48955bbb23a089fec3a"} Sep 06 19:27:41.806663 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.806625 2575 generic.go:356] "Generic (PLEG): container finished" podID="7a7a76a58af3f5a735a46f27cbfd95e9" containerID="d6e38724633ee9c232051b751b7292efa714230dce19fbaa3ca08e53ed5eb5fd" exitCode=0 Sep 06 19:27:41.806818 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.806720 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" event={"ID":"7a7a76a58af3f5a735a46f27cbfd95e9","Type":"ContainerDied","Data":"d6e38724633ee9c232051b751b7292efa714230dce19fbaa3ca08e53ed5eb5fd"} Sep 06 19:27:41.813180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.813141 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerStarted","Data":"0be30bbf5691604580057813213dfc65f4ad279710b76285b4bf047fa8e1b4cb"} Sep 06 19:27:41.819642 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.819612 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"317ad6ff42036c38ed5615e555e04442b8a0d967c18b270efb99aa386288ec8e"} Sep 06 19:27:41.824007 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.821747 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-236.ec2.internal" podStartSLOduration=2.821729474 podStartE2EDuration="2.821729474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:41.806306756 +0000 UTC m=+4.540911492" watchObservedRunningTime="2026-09-06 19:27:41.821729474 +0000 UTC m=+4.556334211" Sep 06 19:27:41.833271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.833241 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jbbtr" event={"ID":"052d0c66-9223-419b-980f-e592d8e633f0","Type":"ContainerStarted","Data":"cb3d94cbb2cbcc9c9734221ae8e97ecf01c10700fc69f981150fb5fab0e62127"} Sep 06 19:27:41.849668 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.849632 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pfljw" event={"ID":"47b1c86e-16a0-4f33-8436-1b186e573396","Type":"ContainerStarted","Data":"e3179d3324732efe9477d41b53fa62e1060409952d49ea28d92bd691c2a9ede8"} Sep 06 19:27:41.852669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.852640 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-th522" event={"ID":"24a08ec0-deb6-41e9-93ff-8234c06fd1c4","Type":"ContainerStarted","Data":"03892c0bf9442d3c2384cb28200f79198142319c54b8609dd8b331df32fb34b6"} Sep 06 19:27:41.855710 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.855681 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qnljk" event={"ID":"ae7b9294-acfa-4b8d-88af-91860510096d","Type":"ContainerStarted","Data":"37076fcd71d4bfd8b1790bd675d95927c0a50136dc41be04526579090e82df45"} Sep 06 19:27:41.860928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.860901 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v7nd9" event={"ID":"349c8685-ef1a-4e22-b5d4-20fc72aac46a","Type":"ContainerStarted","Data":"2aed667fc6f986f53c01d13524042f84414b8e478fae712c979beb9f600f0d23"} Sep 06 19:27:41.870576 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.868335 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzghj" event={"ID":"79361bdc-daa9-4602-8bdb-e9d0c029546f","Type":"ContainerStarted","Data":"01ed91d549350d2d72305bd7119b172e3071ac1a0e638c5020e2e166bb395b18"} Sep 06 19:27:41.877936 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:41.877910 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" event={"ID":"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605","Type":"ContainerStarted","Data":"ef7fbb5e3bf92377caf93a4065c4723cefc9a6516ae23cca3d2f5d6e13182e8a"} Sep 06 19:27:42.756853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:42.756552 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:42.757358 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:42.756987 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:42.757358 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:42.757079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:42.757358 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:42.757196 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:42.896989 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:42.896904 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" event={"ID":"7a7a76a58af3f5a735a46f27cbfd95e9","Type":"ContainerStarted","Data":"cb6d6017bbcc08ad79252e9fcb00cba1fe3c2dd811798f7f990d42ee97d01506"} Sep 06 19:27:43.374566 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:43.374493 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:43.374755 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:43.374594 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:43.374755 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.374732 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:43.374921 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.374816 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:27:47.374775936 +0000 UTC m=+10.109380652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:43.375388 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.375233 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:43.375388 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.375286 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:27:47.375270401 +0000 UTC m=+10.109875128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:43.475153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:43.475081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:43.475352 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.475264 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:27:43.475352 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.475287 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:27:43.475352 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.475301 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:43.475560 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.475362 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:27:47.475343275 +0000 UTC m=+10.209948004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:43.764554 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:43.764427 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:43.765055 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:43.764571 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:44.756668 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:44.756633 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:44.756668 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:44.756661 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:44.756943 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:44.756804 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:44.756943 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:44.756930 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:45.761971 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:45.761940 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:45.762417 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:45.762072 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:46.756608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:46.756518 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:46.756817 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:46.756518 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:46.756817 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:46.756677 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:46.756934 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:46.756819 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:47.410588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:47.410548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:47.411064 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:47.410633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:47.411064 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.410725 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:47.411064 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.410743 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:47.411064 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.410811 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:27:55.410791444 +0000 UTC m=+18.145396167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:47.411064 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.410831 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:27:55.410821424 +0000 UTC m=+18.145426140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:47.511043 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:47.511013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:47.511498 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.511470 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:27:47.511621 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.511500 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:27:47.511621 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.511517 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:47.511621 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.511590 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:27:55.511570038 +0000 UTC m=+18.246174765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:47.757297 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:47.757218 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:47.757451 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:47.757350 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:48.757087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:48.756416 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:48.757087 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:48.756543 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:48.757087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:48.756936 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:48.757087 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:48.757037 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:49.758942 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:49.756790 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:49.758942 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:49.756920 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:50.756781 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:50.756387 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:50.756781 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:50.756399 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:50.756781 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:50.756526 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:50.756781 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:50.756592 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:51.756696 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:51.756664 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:51.757152 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:51.756801 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:52.756802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:52.756764 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:52.757279 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:52.756783 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:52.757279 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:52.756899 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:52.757279 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:52.757025 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:53.756290 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:53.756251 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:53.756464 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:53.756398 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:54.756169 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:54.756137 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:54.756169 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:54.756159 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:54.756671 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:54.756262 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:54.756671 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:54.756385 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:55.468648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:55.468611 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:55.468858 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:55.468674 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:55.468858 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.468720 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:55.468858 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.468803 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:11.468787221 +0000 UTC m=+34.203391935 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:27:55.468858 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.468807 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:55.468858 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.468852 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:28:11.468838325 +0000 UTC m=+34.203443035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:27:55.569387 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:55.569348 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:55.569563 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.569493 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:27:55.569563 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.569519 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:27:55.569563 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.569532 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:55.569712 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.569595 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:11.569576366 +0000 UTC m=+34.304181086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:27:55.756140 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:55.756065 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:55.756280 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:55.756192 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:56.756074 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:56.756035 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:56.756259 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:56.756077 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:56.756259 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:56.756163 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:56.756636 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:56.756291 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:57.759091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:57.759061 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:57.759486 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:57.759162 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:58.757013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:58.756978 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:27:58.757175 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:58.757109 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:27:58.757175 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:58.757150 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:27:58.757312 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:58.757251 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:27:59.756189 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.756022 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:27:59.756912 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:27:59.756286 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:27:59.935332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.935258 2575 generic.go:356] "Generic (PLEG): container finished" podID="16b7f01c-8c0b-4b1d-944e-9ca9418371b3" containerID="b805d89bbb31bd5507ff41f475612b6015aedfab9ca3fbf9a5ed08d8e13854fa" exitCode=0 Sep 06 19:27:59.935563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.935345 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerDied","Data":"b805d89bbb31bd5507ff41f475612b6015aedfab9ca3fbf9a5ed08d8e13854fa"} Sep 06 19:27:59.939674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.939444 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:27:59.940198 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940173 2575 generic.go:356] "Generic (PLEG): container finished" podID="4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3" containerID="456d50a3a08a89b0aa19076df9c6106c66de6fb6e0ca14252f54c8223310e7e6" exitCode=1 Sep 06 19:27:59.940319 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940260 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"254e711353bd75cab0cbe7c4eef7c92dba157d0cc1c278917b43ddf9ec2607af"} Sep 06 19:27:59.940386 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940339 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"d339bf2e74431f9aaa7099ee9d184057f1306a5e28a23d92256839251e47f4b3"} Sep 06 19:27:59.940447 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940392 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"6ce36099e7b3a904d024325ae386444c79b04ea020c6f52e638646f6952de6e3"} Sep 06 19:27:59.940447 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940441 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"4e40a3b141cbf9f95358ec9d758f77c4c2202703b92a9e6e2c7f6e9ac66be938"} Sep 06 19:27:59.940546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerDied","Data":"456d50a3a08a89b0aa19076df9c6106c66de6fb6e0ca14252f54c8223310e7e6"} Sep 06 19:27:59.940546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.940538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"eee482036d0394a63516bc0a2fefa606aa5e376984dbce76c0f2a1571799044f"} Sep 06 19:27:59.942057 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.942028 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jbbtr" event={"ID":"052d0c66-9223-419b-980f-e592d8e633f0","Type":"ContainerStarted","Data":"a355c9b67dde502c5487f1f34700a2f67632f6c94f626d8a2fe4755e4311b942"} Sep 06 19:27:59.943760 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.943723 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pfljw" event={"ID":"47b1c86e-16a0-4f33-8436-1b186e573396","Type":"ContainerStarted","Data":"7c3836770070ef2140e4525a60ecb2e141dc5f6e23569c3b0b9f8cc896b2da51"} Sep 06 19:27:59.946564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.945738 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-th522" event={"ID":"24a08ec0-deb6-41e9-93ff-8234c06fd1c4","Type":"ContainerStarted","Data":"359ac0689f75d65d9ba3cf2105301f1929e26436a6040da8101684e6ad36f7b4"} Sep 06 19:27:59.948553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.948139 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qnljk" event={"ID":"ae7b9294-acfa-4b8d-88af-91860510096d","Type":"ContainerStarted","Data":"6c0f109131d8962b6d213e7cfaab813e8d338fe07b4d61aa60280eca59a5f8ec"} Sep 06 19:27:59.950091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.950067 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v7nd9" event={"ID":"349c8685-ef1a-4e22-b5d4-20fc72aac46a","Type":"ContainerStarted","Data":"34a3f75ffa8627fef8bb73cbfc884f0463af5967bf976e0fa4891d52f8fae067"} Sep 06 19:27:59.951839 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.951816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" event={"ID":"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605","Type":"ContainerStarted","Data":"395f24148813ea6344f869775a5dbd2c34aa37acce43d6ec2a377fcc9d047002"} Sep 06 19:27:59.954973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.954936 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-236.ec2.internal" podStartSLOduration=20.954921114 podStartE2EDuration="20.954921114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:42.921072229 +0000 UTC m=+5.655676967" watchObservedRunningTime="2026-09-06 19:27:59.954921114 +0000 UTC m=+22.689525846" Sep 06 19:27:59.969561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.968850 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-th522" podStartSLOduration=3.911947269 podStartE2EDuration="21.968828587s" totalImagesPullingTime="18.056881318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:59.96740605 +0000 UTC m=+22.702010783" watchObservedRunningTime="2026-09-06 19:27:59.968828587 +0000 UTC m=+22.703433321" Sep 06 19:27:59.980903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:27:59.980857 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qnljk" podStartSLOduration=8.016206251 podStartE2EDuration="21.980837435s" totalImagesPullingTime="13.964631184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:59.98031551 +0000 UTC m=+22.714920243" watchObservedRunningTime="2026-09-06 19:27:59.980837435 +0000 UTC m=+22.715442167" Sep 06 19:28:00.000230 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.000178 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pfljw" podStartSLOduration=3.901943296 podStartE2EDuration="22.000163202s" totalImagesPullingTime="18.098219906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:27:59.999910805 +0000 UTC m=+22.734515550" watchObservedRunningTime="2026-09-06 19:28:00.000163202 +0000 UTC m=+22.734767934" Sep 06 19:28:00.028257 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.028207 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v7nd9" podStartSLOduration=3.899537907 podStartE2EDuration="22.028192357s" totalImagesPullingTime="18.12865445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:00.015409754 +0000 UTC m=+22.750014485" watchObservedRunningTime="2026-09-06 19:28:00.028192357 +0000 UTC m=+22.762797089" Sep 06 19:28:00.028588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.028563 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jbbtr" podStartSLOduration=3.9344523689999997 podStartE2EDuration="22.028554665s" totalImagesPullingTime="18.094102296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:00.02824891 +0000 UTC m=+22.762853642" watchObservedRunningTime="2026-09-06 19:28:00.028554665 +0000 UTC m=+22.763159397" Sep 06 19:28:00.714526 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.714476 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 06 19:28:00.756355 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.756320 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:00.756778 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:00.756450 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:00.756778 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.756514 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:00.756778 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:00.756654 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:00.956513 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.956420 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzghj" event={"ID":"79361bdc-daa9-4602-8bdb-e9d0c029546f","Type":"ContainerStarted","Data":"5ecc0219884d68b4ddd4afd18cac43440f19085b404a3e680582bcbbf9a692e7"} Sep 06 19:28:00.958612 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:00.958580 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" event={"ID":"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605","Type":"ContainerStarted","Data":"44a0984ae45cf1eb6069f0a80b34008c8b478868996a314eaec8985f328033b9"} Sep 06 19:28:01.689459 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.689364 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T19:28:00.714497648Z","UUID":"19d9ff69-90fd-4638-862c-d3477623e1de","Handler":null,"Name":"","Endpoint":""} Sep 06 19:28:01.691058 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.691033 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 06 19:28:01.691176 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.691075 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 06 19:28:01.757006 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.756975 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:01.757474 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:01.757094 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:01.963169 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.963135 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" event={"ID":"8bf1148c-8d55-47ee-b7cb-2ac95c6fc605","Type":"ContainerStarted","Data":"c1833f8bafb3894c1891fa52b497f3d946da161c0cdb349237e5eeaa78482fea"} Sep 06 19:28:01.978231 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.978179 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8glqc" podStartSLOduration=3.793272472 podStartE2EDuration="23.978159038s" totalImagesPullingTime="20.184886566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:01.97811063 +0000 UTC m=+24.712715363" watchObservedRunningTime="2026-09-06 19:28:01.978159038 +0000 UTC m=+24.712763771" Sep 06 19:28:01.978726 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:01.978686 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mzghj" podStartSLOduration=5.927503358 podStartE2EDuration="23.978672645s" totalImagesPullingTime="18.051169287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:00.969266029 +0000 UTC m=+23.703870760" watchObservedRunningTime="2026-09-06 19:28:01.978672645 +0000 UTC m=+24.713277379" Sep 06 19:28:02.674995 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.674949 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:28:02.675613 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.675591 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:28:02.756853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.756818 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:02.757042 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:02.756942 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:02.757042 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.756966 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:02.757425 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:02.757086 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:02.968771 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.968685 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:28:02.969252 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.969215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"c49cb71abd07c16c79323c616d2114aa88d310cd35d214c57fdceaf0a016a557"} Sep 06 19:28:02.969603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.969587 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:28:02.970012 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:02.969988 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qnljk" Sep 06 19:28:03.756367 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:03.756332 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:03.756533 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:03.756473 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:04.756385 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.756156 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:04.757069 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.756198 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:04.757069 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:04.756425 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:04.757069 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:04.756489 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:04.975345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.975304 2575 generic.go:356] "Generic (PLEG): container finished" podID="16b7f01c-8c0b-4b1d-944e-9ca9418371b3" containerID="97117235efab7244051da5b0e8ca1d68724a13d32d374d7b7ec8752bb7567ba7" exitCode=0 Sep 06 19:28:04.975490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.975363 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerDied","Data":"97117235efab7244051da5b0e8ca1d68724a13d32d374d7b7ec8752bb7567ba7"} Sep 06 19:28:04.979201 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.979183 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:28:04.979648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.979624 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"1c581a51ecf750607ad09140e76179dc9f1c0bdf61539f2a84b58cbc4c5f1906"} Sep 06 19:28:04.980082 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:04.980065 2575 scope.go:117] "RemoveContainer" containerID="456d50a3a08a89b0aa19076df9c6106c66de6fb6e0ca14252f54c8223310e7e6" Sep 06 19:28:05.756399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.756345 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:05.756855 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:05.756491 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:05.984787 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.984501 2575 generic.go:356] "Generic (PLEG): container finished" podID="16b7f01c-8c0b-4b1d-944e-9ca9418371b3" containerID="0a66331a95c834f951ad5149f9945f5b78dda1575b005f7ab3ef15e5d9b431dd" exitCode=0 Sep 06 19:28:05.984787 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.984579 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerDied","Data":"0a66331a95c834f951ad5149f9945f5b78dda1575b005f7ab3ef15e5d9b431dd"} Sep 06 19:28:05.991023 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.991000 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:28:05.991956 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.991928 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" event={"ID":"4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3","Type":"ContainerStarted","Data":"f396a067325d6a1a0c73d492813da5aa5b08d5d176d9d0c288fced720c6d8d05"} Sep 06 19:28:05.992044 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.992013 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 06 19:28:05.992207 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.992187 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:28:05.992268 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:05.992261 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:28:06.016067 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.016024 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:28:06.018906 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.018883 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:28:06.026617 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.026571 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" podStartSLOduration=9.927181885 podStartE2EDuration="28.026557506s" totalImagesPullingTime="18.099375621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:06.026530612 +0000 UTC m=+28.761135345" watchObservedRunningTime="2026-09-06 19:28:06.026557506 +0000 UTC m=+28.761162215" Sep 06 19:28:06.250477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.250355 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p2qnt"] Sep 06 19:28:06.250627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.250500 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:06.250627 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:06.250602 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:06.252886 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.252845 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9j4ml"] Sep 06 19:28:06.253047 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.253015 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:06.253155 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:06.253122 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:06.262488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.262464 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4f985"] Sep 06 19:28:06.262611 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.262567 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:06.262693 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:06.262676 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:06.997020 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.996925 2575 generic.go:356] "Generic (PLEG): container finished" podID="16b7f01c-8c0b-4b1d-944e-9ca9418371b3" containerID="42f823afbab1e733a4e0ffe217857710860d23fe40fca1cc0160061c803a4705" exitCode=0 Sep 06 19:28:06.997020 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.997011 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerDied","Data":"42f823afbab1e733a4e0ffe217857710860d23fe40fca1cc0160061c803a4705"} Sep 06 19:28:06.997466 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:06.997066 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 06 19:28:07.757124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:07.757085 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:07.757297 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:07.757188 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:07.757297 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:07.757269 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:07.757423 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:07.757296 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:07.757423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:07.757337 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:07.757511 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:07.757424 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:08.000068 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:08.000029 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 06 19:28:09.046045 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.046013 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:28:09.046465 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.046273 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 06 19:28:09.058627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.058573 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" podUID="4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3" containerName="ovnkube-controller" probeResult="failure" output="" Sep 06 19:28:09.069018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.068984 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" podUID="4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3" containerName="ovnkube-controller" probeResult="failure" output="" Sep 06 19:28:09.756484 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.756450 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:09.756656 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:09.756582 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:09.756727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.756658 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:09.756821 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:09.756801 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:09.756879 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:09.756446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:09.756926 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:09.756899 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:11.484026 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:11.483778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:11.484026 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:11.483996 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:11.484483 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.483939 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 19:28:11.484483 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.484093 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:28:11.484483 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.484096 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret podName:4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:43.484080058 +0000 UTC m=+66.218684767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret") pod "global-pull-secret-syncer-4f985" (UID: "4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4") : object "kube-system"/"original-pull-secret" not registered Sep 06 19:28:11.484483 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.484142 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs podName:46414df0-60db-46f7-91d2-5378dff237aa nodeName:}" failed. No retries permitted until 2026-09-06 19:28:43.484131847 +0000 UTC m=+66.218736557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs") pod "network-metrics-daemon-p2qnt" (UID: "46414df0-60db-46f7-91d2-5378dff237aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 19:28:11.584328 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:11.584286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:11.584505 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.584410 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 19:28:11.584505 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.584426 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 19:28:11.584505 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.584435 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5hs6l for pod openshift-network-diagnostics/network-check-target-9j4ml: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:28:11.584505 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.584484 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l podName:784197f3-23f3-478d-9314-c372025c1a2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:43.584471635 +0000 UTC m=+66.319076344 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hs6l" (UniqueName: "kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l") pod "network-check-target-9j4ml" (UID: "784197f3-23f3-478d-9314-c372025c1a2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 19:28:11.756121 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:11.756039 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:11.756281 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:11.756050 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:11.756281 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.756139 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4f985" podUID="4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4" Sep 06 19:28:11.756281 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:11.756166 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:11.756281 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.756269 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p2qnt" podUID="46414df0-60db-46f7-91d2-5378dff237aa" Sep 06 19:28:11.756488 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:11.756362 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9j4ml" podUID="784197f3-23f3-478d-9314-c372025c1a2a" Sep 06 19:28:12.009737 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.009654 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-236.ec2.internal" event="NodeReady" Sep 06 19:28:12.009916 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.009796 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 06 19:28:12.013345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.013319 2575 generic.go:356] "Generic (PLEG): container finished" podID="16b7f01c-8c0b-4b1d-944e-9ca9418371b3" containerID="382b6f062dba6d218be5dc5fa9cb8c993bc21e9926e32ef490e12b29e9bc9984" exitCode=0 Sep 06 19:28:12.013458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.013382 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerDied","Data":"382b6f062dba6d218be5dc5fa9cb8c993bc21e9926e32ef490e12b29e9bc9984"} Sep 06 19:28:12.052180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.052152 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-48bqg"] Sep 06 19:28:12.056420 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.056398 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v49bv"] Sep 06 19:28:12.056734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.056619 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.059174 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.059149 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 06 19:28:12.059341 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.059323 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 06 19:28:12.059557 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.059339 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kglk6\"" Sep 06 19:28:12.059844 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.059826 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.062340 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.062320 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 06 19:28:12.062340 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.062322 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 06 19:28:12.062484 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.062361 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 06 19:28:12.062561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.062553 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8xksm\"" Sep 06 19:28:12.064283 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.064259 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-48bqg"] Sep 06 19:28:12.066945 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.066925 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v49bv"] Sep 06 19:28:12.188464 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.188433 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8jq\" (UniqueName: \"kubernetes.io/projected/cb875ad2-e098-4759-b872-1ed9f3e2fabd-kube-api-access-vw8jq\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.188572 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.188476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb875ad2-e098-4759-b872-1ed9f3e2fabd-config-volume\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.188572 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.188518 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb875ad2-e098-4759-b872-1ed9f3e2fabd-tmp-dir\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.188572 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.188537 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmpf4\" (UniqueName: \"kubernetes.io/projected/fd1bce41-2076-4298-879d-14c5ac56a61c-kube-api-access-zmpf4\") pod \"ingress-canary-v49bv\" (UID: \"fd1bce41-2076-4298-879d-14c5ac56a61c\") " pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.188572 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.188559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.188693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.188600 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd1bce41-2076-4298-879d-14c5ac56a61c-cert\") pod \"ingress-canary-v49bv\" (UID: \"fd1bce41-2076-4298-879d-14c5ac56a61c\") " pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.289720 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.289646 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.289720 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.289687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd1bce41-2076-4298-879d-14c5ac56a61c-cert\") pod \"ingress-canary-v49bv\" (UID: \"fd1bce41-2076-4298-879d-14c5ac56a61c\") " pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.289874 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.289743 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8jq\" (UniqueName: \"kubernetes.io/projected/cb875ad2-e098-4759-b872-1ed9f3e2fabd-kube-api-access-vw8jq\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.289874 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.289812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb875ad2-e098-4759-b872-1ed9f3e2fabd-config-volume\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.289874 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.289848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb875ad2-e098-4759-b872-1ed9f3e2fabd-tmp-dir\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.289874 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:12.289865 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 19:28:12.290019 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.289878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmpf4\" (UniqueName: \"kubernetes.io/projected/fd1bce41-2076-4298-879d-14c5ac56a61c-kube-api-access-zmpf4\") pod \"ingress-canary-v49bv\" (UID: \"fd1bce41-2076-4298-879d-14c5ac56a61c\") " pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.290019 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:12.289920 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls podName:cb875ad2-e098-4759-b872-1ed9f3e2fabd nodeName:}" failed. No retries permitted until 2026-09-06 19:28:12.789905475 +0000 UTC m=+35.524510185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls") pod "dns-default-48bqg" (UID: "cb875ad2-e098-4759-b872-1ed9f3e2fabd") : secret "dns-default-metrics-tls" not found Sep 06 19:28:12.290308 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.290285 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb875ad2-e098-4759-b872-1ed9f3e2fabd-tmp-dir\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.290434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.290418 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb875ad2-e098-4759-b872-1ed9f3e2fabd-config-volume\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.294050 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.294033 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd1bce41-2076-4298-879d-14c5ac56a61c-cert\") pod \"ingress-canary-v49bv\" (UID: \"fd1bce41-2076-4298-879d-14c5ac56a61c\") " pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.298407 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.298388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmpf4\" (UniqueName: \"kubernetes.io/projected/fd1bce41-2076-4298-879d-14c5ac56a61c-kube-api-access-zmpf4\") pod \"ingress-canary-v49bv\" (UID: \"fd1bce41-2076-4298-879d-14c5ac56a61c\") " pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.298614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.298597 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8jq\" (UniqueName: \"kubernetes.io/projected/cb875ad2-e098-4759-b872-1ed9f3e2fabd-kube-api-access-vw8jq\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.390852 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.390811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v49bv" Sep 06 19:28:12.664256 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.664185 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v49bv"] Sep 06 19:28:12.668044 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:12.668019 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1bce41_2076_4298_879d_14c5ac56a61c.slice/crio-b1bd2bedce1e143e1bc8247d8b2a56db275b2af711b0254cf749219dc1c65574 WatchSource:0}: Error finding container b1bd2bedce1e143e1bc8247d8b2a56db275b2af711b0254cf749219dc1c65574: Status 404 returned error can't find the container with id b1bd2bedce1e143e1bc8247d8b2a56db275b2af711b0254cf749219dc1c65574 Sep 06 19:28:12.793656 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:12.793619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:12.793846 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:12.793731 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 19:28:12.793846 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:12.793804 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls podName:cb875ad2-e098-4759-b872-1ed9f3e2fabd nodeName:}" failed. No retries permitted until 2026-09-06 19:28:13.793790332 +0000 UTC m=+36.528395055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls") pod "dns-default-48bqg" (UID: "cb875ad2-e098-4759-b872-1ed9f3e2fabd") : secret "dns-default-metrics-tls" not found Sep 06 19:28:13.018401 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.018318 2575 generic.go:356] "Generic (PLEG): container finished" podID="16b7f01c-8c0b-4b1d-944e-9ca9418371b3" containerID="4eabc04688c574601e3053ddb8fa408e5538889fcb5387d5a000c9353d818ae2" exitCode=0 Sep 06 19:28:13.018401 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.018389 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerDied","Data":"4eabc04688c574601e3053ddb8fa408e5538889fcb5387d5a000c9353d818ae2"} Sep 06 19:28:13.019593 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.019568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v49bv" event={"ID":"fd1bce41-2076-4298-879d-14c5ac56a61c","Type":"ContainerStarted","Data":"b1bd2bedce1e143e1bc8247d8b2a56db275b2af711b0254cf749219dc1c65574"} Sep 06 19:28:13.756599 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.756341 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:13.757128 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.756342 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:13.757128 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.756388 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:13.759699 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.759223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-26tb5\"" Sep 06 19:28:13.759699 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.759354 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 06 19:28:13.759699 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.759448 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 06 19:28:13.759699 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.759544 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 06 19:28:13.759699 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.759617 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 06 19:28:13.760412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.760390 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bdz9\"" Sep 06 19:28:13.800846 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:13.800821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:13.800978 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:13.800963 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 19:28:13.801029 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:13.801024 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls podName:cb875ad2-e098-4759-b872-1ed9f3e2fabd nodeName:}" failed. No retries permitted until 2026-09-06 19:28:15.801008445 +0000 UTC m=+38.535613158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls") pod "dns-default-48bqg" (UID: "cb875ad2-e098-4759-b872-1ed9f3e2fabd") : secret "dns-default-metrics-tls" not found Sep 06 19:28:14.026194 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:14.026100 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" event={"ID":"16b7f01c-8c0b-4b1d-944e-9ca9418371b3","Type":"ContainerStarted","Data":"3b20be51a732c3cacec6cbbfb65f6077c7c7d82c2c80f53a8b678f87a11e963f"} Sep 06 19:28:14.045963 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:14.045899 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hmmz8" podStartSLOduration=8.216484603 podStartE2EDuration="36.045881809s" totalImagesPullingTime="27.829397206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:14.045868606 +0000 UTC m=+36.780473340" watchObservedRunningTime="2026-09-06 19:28:14.045881809 +0000 UTC m=+36.780486540" Sep 06 19:28:15.031797 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:15.031739 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v49bv" event={"ID":"fd1bce41-2076-4298-879d-14c5ac56a61c","Type":"ContainerStarted","Data":"b30b701628e3624f2b032e6cab0d86eefbe8978436278f02cf8371dfb6f05088"} Sep 06 19:28:15.048944 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:15.048889 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v49bv" podStartSLOduration=0.790332409 podStartE2EDuration="3.048867185s" totalImagesPullingTime="2.258534776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:15.048430657 +0000 UTC m=+37.783035390" watchObservedRunningTime="2026-09-06 19:28:15.048867185 +0000 UTC m=+37.783471919" Sep 06 19:28:15.817539 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:15.817502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:15.817691 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:15.817632 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 19:28:15.817691 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:15.817684 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls podName:cb875ad2-e098-4759-b872-1ed9f3e2fabd nodeName:}" failed. No retries permitted until 2026-09-06 19:28:19.817670185 +0000 UTC m=+42.552274895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls") pod "dns-default-48bqg" (UID: "cb875ad2-e098-4759-b872-1ed9f3e2fabd") : secret "dns-default-metrics-tls" not found Sep 06 19:28:19.847541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:19.847490 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:19.847944 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:19.847643 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 19:28:19.847944 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:19.847709 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls podName:cb875ad2-e098-4759-b872-1ed9f3e2fabd nodeName:}" failed. No retries permitted until 2026-09-06 19:28:27.847691086 +0000 UTC m=+50.582295796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls") pod "dns-default-48bqg" (UID: "cb875ad2-e098-4759-b872-1ed9f3e2fabd") : secret "dns-default-metrics-tls" not found Sep 06 19:28:21.405773 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.405722 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-sj2fc"] Sep 06 19:28:21.441122 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.441085 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj"] Sep 06 19:28:21.441297 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.441224 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" Sep 06 19:28:21.445335 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.445310 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6d9ld\"" Sep 06 19:28:21.454808 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.454785 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-sj2fc"] Sep 06 19:28:21.454941 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.454823 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj"] Sep 06 19:28:21.454941 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.454860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.457043 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.457020 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-t5jbb\"" Sep 06 19:28:21.457150 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.457059 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 06 19:28:21.457241 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.457208 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 06 19:28:21.457309 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.457265 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 06 19:28:21.457309 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.457290 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 06 19:28:21.511404 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.511365 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-x6nsc"] Sep 06 19:28:21.528625 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.528593 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dblnk"] Sep 06 19:28:21.528823 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.528742 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:21.531036 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.531015 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4m8dv\"" Sep 06 19:28:21.531321 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.531299 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 06 19:28:21.531494 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.531355 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 06 19:28:21.554766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.554717 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6bc94b9958-gc6br"] Sep 06 19:28:21.554907 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.554832 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.557228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.557201 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 06 19:28:21.557357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.557281 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 06 19:28:21.557357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.557317 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 06 19:28:21.557482 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.557435 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-d4wbs\"" Sep 06 19:28:21.557832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.557816 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 06 19:28:21.558179 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.558145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zk4t\" (UniqueName: \"kubernetes.io/projected/589ac648-c96c-48bd-905e-b38163649f2e-kube-api-access-2zk4t\") pod \"network-check-source-685456655f-sj2fc\" (UID: \"589ac648-c96c-48bd-905e-b38163649f2e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" Sep 06 19:28:21.558245 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.558175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlr5t\" (UniqueName: \"kubernetes.io/projected/09828082-8a90-466f-85ff-02776367a618-kube-api-access-jlr5t\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.558293 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.558266 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09828082-8a90-466f-85ff-02776367a618-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.558402 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.558303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09828082-8a90-466f-85ff-02776367a618-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.562648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.562629 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 06 19:28:21.569048 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.569031 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-x6nsc"] Sep 06 19:28:21.569191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.569060 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dblnk"] Sep 06 19:28:21.569191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.569081 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6bc94b9958-gc6br"] Sep 06 19:28:21.569191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.569146 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.571301 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571281 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 06 19:28:21.571301 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571296 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 06 19:28:21.571561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571542 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 06 19:28:21.571676 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571662 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 06 19:28:21.571721 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571666 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 06 19:28:21.571721 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571689 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gz27p\"" Sep 06 19:28:21.571813 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.571787 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 06 19:28:21.659703 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659628 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15c8c28-2067-439e-b876-0575479656de-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.659703 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659659 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d15c8c28-2067-439e-b876-0575479656de-serving-cert\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.659703 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zk4t\" (UniqueName: \"kubernetes.io/projected/589ac648-c96c-48bd-905e-b38163649f2e-kube-api-access-2zk4t\") pod \"network-check-source-685456655f-sj2fc\" (UID: \"589ac648-c96c-48bd-905e-b38163649f2e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" Sep 06 19:28:21.659921 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659721 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlr5t\" (UniqueName: \"kubernetes.io/projected/09828082-8a90-466f-85ff-02776367a618-kube-api-access-jlr5t\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.659921 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659737 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:21.659921 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.659921 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659897 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.660037 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd9xj\" (UniqueName: \"kubernetes.io/projected/34fe05ee-374e-477f-96b3-df13f2182b01-kube-api-access-nd9xj\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.660037 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.659941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d15c8c28-2067-439e-b876-0575479656de-snapshots\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.660037 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d15c8c28-2067-439e-b876-0575479656de-tmp\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.660125 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660049 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-stats-auth\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.660125 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15c8c28-2067-439e-b876-0575479656de-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.660125 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09828082-8a90-466f-85ff-02776367a618-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.660217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660128 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09828082-8a90-466f-85ff-02776367a618-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.660217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-nginx-conf\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:21.660217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-default-certificate\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.660217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660177 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6296\" (UniqueName: \"kubernetes.io/projected/d15c8c28-2067-439e-b876-0575479656de-kube-api-access-b6296\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.660703 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.660676 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09828082-8a90-466f-85ff-02776367a618-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.663726 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.663707 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09828082-8a90-466f-85ff-02776367a618-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.667870 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.667849 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zk4t\" (UniqueName: \"kubernetes.io/projected/589ac648-c96c-48bd-905e-b38163649f2e-kube-api-access-2zk4t\") pod \"network-check-source-685456655f-sj2fc\" (UID: \"589ac648-c96c-48bd-905e-b38163649f2e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" Sep 06 19:28:21.667952 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.667891 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlr5t\" (UniqueName: \"kubernetes.io/projected/09828082-8a90-466f-85ff-02776367a618-kube-api-access-jlr5t\") pod \"kube-storage-version-migrator-operator-8d7dd4675-d7hpj\" (UID: \"09828082-8a90-466f-85ff-02776367a618\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.752107 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.752077 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" Sep 06 19:28:21.762391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.762353 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:21.762843 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.762825 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.762933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.762849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.762933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.762866 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd9xj\" (UniqueName: \"kubernetes.io/projected/34fe05ee-374e-477f-96b3-df13f2182b01-kube-api-access-nd9xj\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.762933 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:21.762490 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 19:28:21.762933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.762847 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" Sep 06 19:28:21.762933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.762894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d15c8c28-2067-439e-b876-0575479656de-snapshots\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:21.762950 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert podName:d1bf5b94-eaf1-4e88-96b2-153f7a967e2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:22.262927696 +0000 UTC m=+44.997532409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-x6nsc" (UID: "d1bf5b94-eaf1-4e88-96b2-153f7a967e2a") : secret "networking-console-plugin-cert" not found Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:21.763002 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:22.262980707 +0000 UTC m=+44.997585422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : configmap references non-existent config key: service-ca.crt Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:21.763027 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d15c8c28-2067-439e-b876-0575479656de-tmp\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:21.763100 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:22.263071252 +0000 UTC m=+44.997675982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : secret "router-metrics-certs-default" not found Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-stats-auth\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.763150 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763156 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15c8c28-2067-439e-b876-0575479656de-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763482 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-nginx-conf\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:21.763482 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-default-certificate\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.763482 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763224 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6296\" (UniqueName: \"kubernetes.io/projected/d15c8c28-2067-439e-b876-0575479656de-kube-api-access-b6296\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763482 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15c8c28-2067-439e-b876-0575479656de-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763482 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d15c8c28-2067-439e-b876-0575479656de-serving-cert\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763700 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763549 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d15c8c28-2067-439e-b876-0575479656de-snapshots\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.763774 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.763713 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15c8c28-2067-439e-b876-0575479656de-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.764066 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.764043 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d15c8c28-2067-439e-b876-0575479656de-tmp\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.764153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.764102 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-nginx-conf\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:21.764226 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.764209 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15c8c28-2067-439e-b876-0575479656de-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.765728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.765706 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d15c8c28-2067-439e-b876-0575479656de-serving-cert\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.766197 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.766174 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-stats-auth\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.766326 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.766303 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-default-certificate\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.770291 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.770269 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd9xj\" (UniqueName: \"kubernetes.io/projected/34fe05ee-374e-477f-96b3-df13f2182b01-kube-api-access-nd9xj\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:21.771166 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.771143 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6296\" (UniqueName: \"kubernetes.io/projected/d15c8c28-2067-439e-b876-0575479656de-kube-api-access-b6296\") pod \"insights-operator-76bcd6ccf5-dblnk\" (UID: \"d15c8c28-2067-439e-b876-0575479656de\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:21.864096 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:21.864066 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" Sep 06 19:28:22.047819 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:22.047786 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-sj2fc"] Sep 06 19:28:22.050890 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:22.050869 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589ac648_c96c_48bd_905e_b38163649f2e.slice/crio-ba826a42aa71b7a21e6c214068ba2bc2fce0942b26d5305325ddd06d89003962 WatchSource:0}: Error finding container ba826a42aa71b7a21e6c214068ba2bc2fce0942b26d5305325ddd06d89003962: Status 404 returned error can't find the container with id ba826a42aa71b7a21e6c214068ba2bc2fce0942b26d5305325ddd06d89003962 Sep 06 19:28:22.071514 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:22.071484 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj"] Sep 06 19:28:22.075166 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:22.075130 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09828082_8a90_466f_85ff_02776367a618.slice/crio-709ee04d268a91a2f1f39b818f119ba5ae7bf383a1f0e4575015b4da81bc6303 WatchSource:0}: Error finding container 709ee04d268a91a2f1f39b818f119ba5ae7bf383a1f0e4575015b4da81bc6303: Status 404 returned error can't find the container with id 709ee04d268a91a2f1f39b818f119ba5ae7bf383a1f0e4575015b4da81bc6303 Sep 06 19:28:22.161585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:22.161542 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dblnk"] Sep 06 19:28:22.165355 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:22.165328 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15c8c28_2067_439e_b876_0575479656de.slice/crio-4eae83a1ce4bb0004c7101d7acf05a3d089d76731e5cd6c89fa201730031be79 WatchSource:0}: Error finding container 4eae83a1ce4bb0004c7101d7acf05a3d089d76731e5cd6c89fa201730031be79: Status 404 returned error can't find the container with id 4eae83a1ce4bb0004c7101d7acf05a3d089d76731e5cd6c89fa201730031be79 Sep 06 19:28:22.267648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:22.267579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:22.267648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:22.267619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:22.267648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:22.267639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:22.267898 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:22.267729 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 19:28:22.267898 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:22.267741 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 19:28:22.267898 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:22.267785 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:23.267735885 +0000 UTC m=+46.002340596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : configmap references non-existent config key: service-ca.crt Sep 06 19:28:22.267898 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:22.267811 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:23.267798847 +0000 UTC m=+46.002403557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : secret "router-metrics-certs-default" not found Sep 06 19:28:22.267898 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:22.267825 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert podName:d1bf5b94-eaf1-4e88-96b2-153f7a967e2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:23.267819273 +0000 UTC m=+46.002423983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-x6nsc" (UID: "d1bf5b94-eaf1-4e88-96b2-153f7a967e2a") : secret "networking-console-plugin-cert" not found Sep 06 19:28:23.054711 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:23.054667 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" event={"ID":"d15c8c28-2067-439e-b876-0575479656de","Type":"ContainerStarted","Data":"4eae83a1ce4bb0004c7101d7acf05a3d089d76731e5cd6c89fa201730031be79"} Sep 06 19:28:23.056407 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:23.056375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" event={"ID":"589ac648-c96c-48bd-905e-b38163649f2e","Type":"ContainerStarted","Data":"ba826a42aa71b7a21e6c214068ba2bc2fce0942b26d5305325ddd06d89003962"} Sep 06 19:28:23.057965 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:23.057897 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" event={"ID":"09828082-8a90-466f-85ff-02776367a618","Type":"ContainerStarted","Data":"709ee04d268a91a2f1f39b818f119ba5ae7bf383a1f0e4575015b4da81bc6303"} Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:23.275791 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:23.275846 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:23.275874 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:23.276059 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:23.276120 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:25.276102497 +0000 UTC m=+48.010707215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : secret "router-metrics-certs-default" not found Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:23.276373 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:25.276352919 +0000 UTC m=+48.010957640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : configmap references non-existent config key: service-ca.crt Sep 06 19:28:23.276468 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:23.276474 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 19:28:23.277001 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:23.276511 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert podName:d1bf5b94-eaf1-4e88-96b2-153f7a967e2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:25.276499754 +0000 UTC m=+48.011104467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-x6nsc" (UID: "d1bf5b94-eaf1-4e88-96b2-153f7a967e2a") : secret "networking-console-plugin-cert" not found Sep 06 19:28:25.292622 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:25.292583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:25.292622 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:25.292629 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:25.293171 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:25.292735 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 19:28:25.293171 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:25.292778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:25.293171 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:25.292817 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert podName:d1bf5b94-eaf1-4e88-96b2-153f7a967e2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:29.292795785 +0000 UTC m=+52.027400513 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-x6nsc" (UID: "d1bf5b94-eaf1-4e88-96b2-153f7a967e2a") : secret "networking-console-plugin-cert" not found Sep 06 19:28:25.293171 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:25.292840 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:29.292833951 +0000 UTC m=+52.027438661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : configmap references non-existent config key: service-ca.crt Sep 06 19:28:25.293171 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:25.292862 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 19:28:25.293171 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:25.292913 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:29.292898813 +0000 UTC m=+52.027503543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : secret "router-metrics-certs-default" not found Sep 06 19:28:27.069577 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.069535 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" event={"ID":"09828082-8a90-466f-85ff-02776367a618","Type":"ContainerStarted","Data":"a0a53d406f1347148e694321cdada26927bc0cc3cc4954ffdfbe172f9929f06e"} Sep 06 19:28:27.071427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.071401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" event={"ID":"d15c8c28-2067-439e-b876-0575479656de","Type":"ContainerStarted","Data":"37557ff79910ae35b07a5ee6850a6e126c995186a9d996d0e12365a2c58f9b38"} Sep 06 19:28:27.073086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.073056 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" event={"ID":"589ac648-c96c-48bd-905e-b38163649f2e","Type":"ContainerStarted","Data":"ca800b02d50a3edd59f73d6b18af19f3fea89046f8aeff3227f8bb9989798da9"} Sep 06 19:28:27.086328 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.086278 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" podStartSLOduration=1.640485493 podStartE2EDuration="6.086263092s" totalImagesPullingTime="4.445777599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:27.085061579 +0000 UTC m=+49.819666319" watchObservedRunningTime="2026-09-06 19:28:27.086263092 +0000 UTC m=+49.820867823" Sep 06 19:28:27.099462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.099407 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-sj2fc" podStartSLOduration=1.6202356770000002 podStartE2EDuration="6.099389187s" totalImagesPullingTime="4.47915351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:27.098201702 +0000 UTC m=+49.832806435" watchObservedRunningTime="2026-09-06 19:28:27.099389187 +0000 UTC m=+49.833993925" Sep 06 19:28:27.115272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.115223 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" podStartSLOduration=1.759623383 podStartE2EDuration="6.11520461s" totalImagesPullingTime="4.355581227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:27.114542408 +0000 UTC m=+49.849147141" watchObservedRunningTime="2026-09-06 19:28:27.11520461 +0000 UTC m=+49.849809346" Sep 06 19:28:27.316346 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.316314 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l6499"] Sep 06 19:28:27.334483 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.334414 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l6499"] Sep 06 19:28:27.334632 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.334544 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.336677 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.336655 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 06 19:28:27.336677 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.336671 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 06 19:28:27.337026 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.337010 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8jvvj\"" Sep 06 19:28:27.512694 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.512660 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99ec35e2-484c-420d-82b0-39b1f63909be-data-volume\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.512884 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.512805 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.512884 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.512853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.512884 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.512879 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99ec35e2-484c-420d-82b0-39b1f63909be-crio-socket\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.513017 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.512893 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5kgf\" (UniqueName: \"kubernetes.io/projected/99ec35e2-484c-420d-82b0-39b1f63909be-kube-api-access-m5kgf\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614197 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614110 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99ec35e2-484c-420d-82b0-39b1f63909be-data-volume\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614358 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614215 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614358 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614242 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614358 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614266 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99ec35e2-484c-420d-82b0-39b1f63909be-crio-socket\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614358 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614291 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5kgf\" (UniqueName: \"kubernetes.io/projected/99ec35e2-484c-420d-82b0-39b1f63909be-kube-api-access-m5kgf\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614358 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:27.614344 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 19:28:27.614590 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:27.614409 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:28.114390441 +0000 UTC m=+50.848995172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : configmap "kube-rbac-proxy" not found Sep 06 19:28:27.614590 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:27.614411 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 19:28:27.614590 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99ec35e2-484c-420d-82b0-39b1f63909be-data-volume\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614590 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.614499 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99ec35e2-484c-420d-82b0-39b1f63909be-crio-socket\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.614590 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:27.614479 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:28.114459496 +0000 UTC m=+50.849064224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : secret "insights-runtime-extractor-tls" not found Sep 06 19:28:27.622526 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.622500 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5kgf\" (UniqueName: \"kubernetes.io/projected/99ec35e2-484c-420d-82b0-39b1f63909be-kube-api-access-m5kgf\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:27.917107 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:27.917070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:27.917287 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:27.917211 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 06 19:28:27.917287 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:27.917285 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls podName:cb875ad2-e098-4759-b872-1ed9f3e2fabd nodeName:}" failed. No retries permitted until 2026-09-06 19:28:43.917267073 +0000 UTC m=+66.651871790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls") pod "dns-default-48bqg" (UID: "cb875ad2-e098-4759-b872-1ed9f3e2fabd") : secret "dns-default-metrics-tls" not found Sep 06 19:28:28.119196 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:28.119161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:28.119196 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:28.119202 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:28.119601 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:28.119281 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 19:28:28.119601 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:28.119341 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:29.119327081 +0000 UTC m=+51.853931790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : configmap "kube-rbac-proxy" not found Sep 06 19:28:28.119601 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:28.119348 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 19:28:28.119601 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:28.119387 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:29.119375983 +0000 UTC m=+51.853980694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : secret "insights-runtime-extractor-tls" not found Sep 06 19:28:29.127479 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:29.127447 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:29.127479 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:29.127484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:29.127925 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.127559 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 19:28:29.127925 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.127570 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 19:28:29.127925 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.127616 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:31.127601896 +0000 UTC m=+53.862206606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : secret "insights-runtime-extractor-tls" not found Sep 06 19:28:29.127925 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.127630 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:31.127622851 +0000 UTC m=+53.862227561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : configmap "kube-rbac-proxy" not found Sep 06 19:28:29.289151 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:29.289123 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jbbtr_052d0c66-9223-419b-980f-e592d8e633f0/dns-node-resolver/0.log" Sep 06 19:28:29.329472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:29.329438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:29.329472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:29.329475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:29.329647 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:29.329497 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:29.329647 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.329585 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 19:28:29.329647 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.329611 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:37.329593731 +0000 UTC m=+60.064198445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : configmap references non-existent config key: service-ca.crt Sep 06 19:28:29.329647 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.329638 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert podName:d1bf5b94-eaf1-4e88-96b2-153f7a967e2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:37.329631577 +0000 UTC m=+60.064236287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-x6nsc" (UID: "d1bf5b94-eaf1-4e88-96b2-153f7a967e2a") : secret "networking-console-plugin-cert" not found Sep 06 19:28:29.329824 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.329664 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 06 19:28:29.329824 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:29.329721 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:37.329703361 +0000 UTC m=+60.064308073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : secret "router-metrics-certs-default" not found Sep 06 19:28:30.288964 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:30.288932 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-th522_24a08ec0-deb6-41e9-93ff-8234c06fd1c4/node-ca/0.log" Sep 06 19:28:31.143933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:31.143899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:31.144140 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:31.143942 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:31.144140 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:31.144049 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 19:28:31.144140 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:31.144090 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 19:28:31.144140 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:31.144123 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:35.144103078 +0000 UTC m=+57.878707805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : configmap "kube-rbac-proxy" not found Sep 06 19:28:31.144310 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:31.144191 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:35.144180482 +0000 UTC m=+57.878785193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : secret "insights-runtime-extractor-tls" not found Sep 06 19:28:31.288391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:31.288355 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v49bv_fd1bce41-2076-4298-879d-14c5ac56a61c/serve-healthcheck-canary/0.log" Sep 06 19:28:35.175660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:35.175612 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:35.175660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:35.175670 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:35.176108 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:35.175736 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 06 19:28:35.176108 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:35.175819 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 06 19:28:35.176108 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:35.175834 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:43.175817909 +0000 UTC m=+65.910422619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : configmap "kube-rbac-proxy" not found Sep 06 19:28:35.176108 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:35.175875 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls podName:99ec35e2-484c-420d-82b0-39b1f63909be nodeName:}" failed. No retries permitted until 2026-09-06 19:28:43.17586177 +0000 UTC m=+65.910466479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6499" (UID: "99ec35e2-484c-420d-82b0-39b1f63909be") : secret "insights-runtime-extractor-tls" not found Sep 06 19:28:37.394660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:37.394619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:37.394660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:37.394657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:37.395112 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:37.394677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:37.395112 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:37.394803 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 06 19:28:37.395112 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:37.394824 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle podName:34fe05ee-374e-477f-96b3-df13f2182b01 nodeName:}" failed. No retries permitted until 2026-09-06 19:28:53.394803972 +0000 UTC m=+76.129408681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle") pod "router-default-6bc94b9958-gc6br" (UID: "34fe05ee-374e-477f-96b3-df13f2182b01") : configmap references non-existent config key: service-ca.crt Sep 06 19:28:37.395112 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:28:37.394862 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert podName:d1bf5b94-eaf1-4e88-96b2-153f7a967e2a nodeName:}" failed. No retries permitted until 2026-09-06 19:28:53.394851618 +0000 UTC m=+76.129456332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-x6nsc" (UID: "d1bf5b94-eaf1-4e88-96b2-153f7a967e2a") : secret "networking-console-plugin-cert" not found Sep 06 19:28:37.397193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:37.397172 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fe05ee-374e-477f-96b3-df13f2182b01-metrics-certs\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:39.068629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:39.068593 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqg62" Sep 06 19:28:43.246875 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.246820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:43.247340 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.246894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:43.247382 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.247352 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99ec35e2-484c-420d-82b0-39b1f63909be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:43.249296 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.249271 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99ec35e2-484c-420d-82b0-39b1f63909be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6499\" (UID: \"99ec35e2-484c-420d-82b0-39b1f63909be\") " pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:43.546355 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.546281 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8jvvj\"" Sep 06 19:28:43.548335 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.548315 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:43.548399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.548368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:43.550428 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.550411 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 06 19:28:43.550486 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.550436 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 06 19:28:43.555240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.555226 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l6499" Sep 06 19:28:43.561791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.561742 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4-original-pull-secret\") pod \"global-pull-secret-syncer-4f985\" (UID: \"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4\") " pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:43.561878 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.561768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46414df0-60db-46f7-91d2-5378dff237aa-metrics-certs\") pod \"network-metrics-daemon-p2qnt\" (UID: \"46414df0-60db-46f7-91d2-5378dff237aa\") " pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:43.649010 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.648980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:43.651680 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.651649 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hs6l\" (UniqueName: \"kubernetes.io/projected/784197f3-23f3-478d-9314-c372025c1a2a-kube-api-access-5hs6l\") pod \"network-check-target-9j4ml\" (UID: \"784197f3-23f3-478d-9314-c372025c1a2a\") " pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:43.772131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.772102 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-26tb5\"" Sep 06 19:28:43.778058 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.778039 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4f985" Sep 06 19:28:43.780175 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.780158 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:43.786320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.786304 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bdz9\"" Sep 06 19:28:43.795157 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.795142 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p2qnt" Sep 06 19:28:43.827354 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.827328 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l6499"] Sep 06 19:28:43.830523 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:43.830499 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ec35e2_484c_420d_82b0_39b1f63909be.slice/crio-e48c75938a0c700293bdfcc97dde0b9e8972e39fb1d1fc0febabe3790212ba5a WatchSource:0}: Error finding container e48c75938a0c700293bdfcc97dde0b9e8972e39fb1d1fc0febabe3790212ba5a: Status 404 returned error can't find the container with id e48c75938a0c700293bdfcc97dde0b9e8972e39fb1d1fc0febabe3790212ba5a Sep 06 19:28:43.951634 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.951604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:43.953938 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:43.953919 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb875ad2-e098-4759-b872-1ed9f3e2fabd-metrics-tls\") pod \"dns-default-48bqg\" (UID: \"cb875ad2-e098-4759-b872-1ed9f3e2fabd\") " pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:44.081243 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.081159 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4f985"] Sep 06 19:28:44.084878 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:44.084852 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4447ca_1b3a_4eca_a32d_ea055bb5cbb4.slice/crio-b4468e8666c75191a098b032675b6508da4124cfec0e55c68af4095f66e21022 WatchSource:0}: Error finding container b4468e8666c75191a098b032675b6508da4124cfec0e55c68af4095f66e21022: Status 404 returned error can't find the container with id b4468e8666c75191a098b032675b6508da4124cfec0e55c68af4095f66e21022 Sep 06 19:28:44.109879 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.109846 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9j4ml"] Sep 06 19:28:44.114480 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:44.114455 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784197f3_23f3_478d_9314_c372025c1a2a.slice/crio-ea4b4cb4fa56622cdb4e09750e8819c88a12850016a7231fc20f84f31631a5e6 WatchSource:0}: Error finding container ea4b4cb4fa56622cdb4e09750e8819c88a12850016a7231fc20f84f31631a5e6: Status 404 returned error can't find the container with id ea4b4cb4fa56622cdb4e09750e8819c88a12850016a7231fc20f84f31631a5e6 Sep 06 19:28:44.115618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.115592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4f985" event={"ID":"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4","Type":"ContainerStarted","Data":"b4468e8666c75191a098b032675b6508da4124cfec0e55c68af4095f66e21022"} Sep 06 19:28:44.117271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.117238 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6499" event={"ID":"99ec35e2-484c-420d-82b0-39b1f63909be","Type":"ContainerStarted","Data":"d674e3b2360e3cb761a01409e3026a6d82a73498a4779f6dbec3abad699fb586"} Sep 06 19:28:44.117356 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.117297 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6499" event={"ID":"99ec35e2-484c-420d-82b0-39b1f63909be","Type":"ContainerStarted","Data":"e48c75938a0c700293bdfcc97dde0b9e8972e39fb1d1fc0febabe3790212ba5a"} Sep 06 19:28:44.136363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.136338 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p2qnt"] Sep 06 19:28:44.140048 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:44.140027 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46414df0_60db_46f7_91d2_5378dff237aa.slice/crio-2b033293f3efb737cc23e354410ed08463dfc98c5e2f0acdce273463b9772184 WatchSource:0}: Error finding container 2b033293f3efb737cc23e354410ed08463dfc98c5e2f0acdce273463b9772184: Status 404 returned error can't find the container with id 2b033293f3efb737cc23e354410ed08463dfc98c5e2f0acdce273463b9772184 Sep 06 19:28:44.187616 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.187593 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kglk6\"" Sep 06 19:28:44.195706 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.195690 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:44.482221 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:44.482191 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-48bqg"] Sep 06 19:28:44.485317 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:44.485284 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb875ad2_e098_4759_b872_1ed9f3e2fabd.slice/crio-ea7198104582b8cb917433c42394c98bcc7573c1e115bde49b93e29afd65808d WatchSource:0}: Error finding container ea7198104582b8cb917433c42394c98bcc7573c1e115bde49b93e29afd65808d: Status 404 returned error can't find the container with id ea7198104582b8cb917433c42394c98bcc7573c1e115bde49b93e29afd65808d Sep 06 19:28:45.124620 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.124516 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6499" event={"ID":"99ec35e2-484c-420d-82b0-39b1f63909be","Type":"ContainerStarted","Data":"461715b0d7efebc367fdcca713530f604926f11d6006580648213c0439e04f94"} Sep 06 19:28:45.126265 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.126197 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p2qnt" event={"ID":"46414df0-60db-46f7-91d2-5378dff237aa","Type":"ContainerStarted","Data":"2b033293f3efb737cc23e354410ed08463dfc98c5e2f0acdce273463b9772184"} Sep 06 19:28:45.130701 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.130563 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9j4ml" event={"ID":"784197f3-23f3-478d-9314-c372025c1a2a","Type":"ContainerStarted","Data":"92feac7f64c1c5c0a80956fc3df22e65ce5e0c2ac82594f1319efcca3d8bfa21"} Sep 06 19:28:45.130701 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.130598 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9j4ml" event={"ID":"784197f3-23f3-478d-9314-c372025c1a2a","Type":"ContainerStarted","Data":"ea4b4cb4fa56622cdb4e09750e8819c88a12850016a7231fc20f84f31631a5e6"} Sep 06 19:28:45.130981 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.130941 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:28:45.133027 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.132978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-48bqg" event={"ID":"cb875ad2-e098-4759-b872-1ed9f3e2fabd","Type":"ContainerStarted","Data":"ea7198104582b8cb917433c42394c98bcc7573c1e115bde49b93e29afd65808d"} Sep 06 19:28:45.144991 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:45.144650 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9j4ml" podStartSLOduration=67.144633105 podStartE2EDuration="1m7.144633105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:45.14309745 +0000 UTC m=+67.877702182" watchObservedRunningTime="2026-09-06 19:28:45.144633105 +0000 UTC m=+67.879237839" Sep 06 19:28:46.141049 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:46.140968 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p2qnt" event={"ID":"46414df0-60db-46f7-91d2-5378dff237aa","Type":"ContainerStarted","Data":"e1e1be5c746ddee904fcbb35f51f0c5b9da7be267be00281a4cff6b9735b3473"} Sep 06 19:28:48.127488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.127451 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r"] Sep 06 19:28:48.130595 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.130573 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.134471 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.134430 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 06 19:28:48.134590 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.134520 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 06 19:28:48.134676 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.134580 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 06 19:28:48.134676 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.134446 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 06 19:28:48.139954 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.139929 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r"] Sep 06 19:28:48.185815 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.185780 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c65f4f9b-whcnh"] Sep 06 19:28:48.186136 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.186114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/39cce6d2-9a58-4bec-9c75-89e3215c09e6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.186213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.186147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39cce6d2-9a58-4bec-9c75-89e3215c09e6-tmp\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.186213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.186176 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7vw9\" (UniqueName: \"kubernetes.io/projected/39cce6d2-9a58-4bec-9c75-89e3215c09e6-kube-api-access-q7vw9\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.188187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.188168 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.190438 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.190414 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sn49m\"" Sep 06 19:28:48.190438 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.190433 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 06 19:28:48.190813 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.190788 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 06 19:28:48.190909 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.190823 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 06 19:28:48.196797 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.195990 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 06 19:28:48.202594 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.201255 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c65f4f9b-whcnh"] Sep 06 19:28:48.286692 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286658 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8643a868-3d64-4e56-8780-305c9d7eeb60-installation-pull-secrets\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.286892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8643a868-3d64-4e56-8780-305c9d7eeb60-image-registry-private-configuration\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.286892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/39cce6d2-9a58-4bec-9c75-89e3215c09e6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.286892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39cce6d2-9a58-4bec-9c75-89e3215c09e6-tmp\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.286892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286856 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7vw9\" (UniqueName: \"kubernetes.io/projected/39cce6d2-9a58-4bec-9c75-89e3215c09e6-kube-api-access-q7vw9\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.286892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8643a868-3d64-4e56-8780-305c9d7eeb60-trusted-ca\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.287111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286920 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8643a868-3d64-4e56-8780-305c9d7eeb60-registry-certificates\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.287111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-registry-tls\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.287111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.286971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdq5\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-kube-api-access-7hdq5\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.287111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.287069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8643a868-3d64-4e56-8780-305c9d7eeb60-ca-trust-extracted\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.287111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.287108 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-bound-sa-token\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.287295 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.287242 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39cce6d2-9a58-4bec-9c75-89e3215c09e6-tmp\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.289661 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.289637 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/39cce6d2-9a58-4bec-9c75-89e3215c09e6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.296314 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.296285 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7vw9\" (UniqueName: \"kubernetes.io/projected/39cce6d2-9a58-4bec-9c75-89e3215c09e6-kube-api-access-q7vw9\") pod \"klusterlet-addon-workmgr-7969fdb55c-n6b5r\" (UID: \"39cce6d2-9a58-4bec-9c75-89e3215c09e6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.388131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388052 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-registry-tls\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388089 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdq5\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-kube-api-access-7hdq5\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388146 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8643a868-3d64-4e56-8780-305c9d7eeb60-ca-trust-extracted\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-bound-sa-token\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388218 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8643a868-3d64-4e56-8780-305c9d7eeb60-installation-pull-secrets\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8643a868-3d64-4e56-8780-305c9d7eeb60-image-registry-private-configuration\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8643a868-3d64-4e56-8780-305c9d7eeb60-trusted-ca\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.388357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.388328 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8643a868-3d64-4e56-8780-305c9d7eeb60-registry-certificates\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.389066 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.389033 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8643a868-3d64-4e56-8780-305c9d7eeb60-ca-trust-extracted\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.389177 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.389082 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8643a868-3d64-4e56-8780-305c9d7eeb60-registry-certificates\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.389955 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.389911 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8643a868-3d64-4e56-8780-305c9d7eeb60-trusted-ca\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.391556 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.391522 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-registry-tls\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.391670 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.391598 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8643a868-3d64-4e56-8780-305c9d7eeb60-image-registry-private-configuration\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.392078 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.392060 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8643a868-3d64-4e56-8780-305c9d7eeb60-installation-pull-secrets\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.401592 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.401561 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdq5\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-kube-api-access-7hdq5\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.401667 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.401621 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8643a868-3d64-4e56-8780-305c9d7eeb60-bound-sa-token\") pod \"image-registry-5c65f4f9b-whcnh\" (UID: \"8643a868-3d64-4e56-8780-305c9d7eeb60\") " pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.442389 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.442354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:48.499951 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.499926 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:48.798730 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.798699 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r"] Sep 06 19:28:48.801661 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:48.801629 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cce6d2_9a58_4bec_9c75_89e3215c09e6.slice/crio-0910fb74fbaaa3460849650f9455d61c54ee9b6594e4746a728afad7f82bf111 WatchSource:0}: Error finding container 0910fb74fbaaa3460849650f9455d61c54ee9b6594e4746a728afad7f82bf111: Status 404 returned error can't find the container with id 0910fb74fbaaa3460849650f9455d61c54ee9b6594e4746a728afad7f82bf111 Sep 06 19:28:48.811250 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:48.811225 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c65f4f9b-whcnh"] Sep 06 19:28:48.814782 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:48.814730 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8643a868_3d64_4e56_8780_305c9d7eeb60.slice/crio-828aca043806eaf208c0f4de851941dd24164038196674096f6f18a2edcb343c WatchSource:0}: Error finding container 828aca043806eaf208c0f4de851941dd24164038196674096f6f18a2edcb343c: Status 404 returned error can't find the container with id 828aca043806eaf208c0f4de851941dd24164038196674096f6f18a2edcb343c Sep 06 19:28:49.157594 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.157550 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" event={"ID":"8643a868-3d64-4e56-8780-305c9d7eeb60","Type":"ContainerStarted","Data":"c990624baea680c57dbefa4bd95ba1086c2cc1e9e706b69e717e96d3208b9329"} Sep 06 19:28:49.158104 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.157607 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" event={"ID":"8643a868-3d64-4e56-8780-305c9d7eeb60","Type":"ContainerStarted","Data":"828aca043806eaf208c0f4de851941dd24164038196674096f6f18a2edcb343c"} Sep 06 19:28:49.158104 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.157790 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:28:49.159312 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.159285 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4f985" event={"ID":"4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4","Type":"ContainerStarted","Data":"ad6be1678933f103f60d042557af34269589d5a9523922ac84f30c71ea9c2a1e"} Sep 06 19:28:49.161169 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.161145 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-48bqg" event={"ID":"cb875ad2-e098-4759-b872-1ed9f3e2fabd","Type":"ContainerStarted","Data":"8db24cafca5d2b953dcf05659819df503f49bcece77e61ea09a705b3a0b2e6c9"} Sep 06 19:28:49.161304 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.161189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-48bqg" event={"ID":"cb875ad2-e098-4759-b872-1ed9f3e2fabd","Type":"ContainerStarted","Data":"d94c6b1de939457379909ee5361f66dceac7bdf3ce67c60a3ce6f160627d9801"} Sep 06 19:28:49.161304 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.161281 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-48bqg" Sep 06 19:28:49.162381 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.162358 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" event={"ID":"39cce6d2-9a58-4bec-9c75-89e3215c09e6","Type":"ContainerStarted","Data":"0910fb74fbaaa3460849650f9455d61c54ee9b6594e4746a728afad7f82bf111"} Sep 06 19:28:49.164509 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.164490 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6499" event={"ID":"99ec35e2-484c-420d-82b0-39b1f63909be","Type":"ContainerStarted","Data":"1ac793fb69822471bdcbc20552fb150c4be61ac647a98aa786d321d5772c8e7a"} Sep 06 19:28:49.166396 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.166368 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p2qnt" event={"ID":"46414df0-60db-46f7-91d2-5378dff237aa","Type":"ContainerStarted","Data":"665a66fec6ac3fece7d28b12d42ef16bfe3fe879e21915ed827cb15ff1b1928f"} Sep 06 19:28:49.176309 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.176268 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" podStartSLOduration=1.176249712 podStartE2EDuration="1.176249712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:49.175404722 +0000 UTC m=+71.910009455" watchObservedRunningTime="2026-09-06 19:28:49.176249712 +0000 UTC m=+71.910854444" Sep 06 19:28:49.189673 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.189612 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4f985" podStartSLOduration=66.753458592 podStartE2EDuration="1m11.189592448s" totalImagesPullingTime="4.436133856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:49.188221873 +0000 UTC m=+71.922826619" watchObservedRunningTime="2026-09-06 19:28:49.189592448 +0000 UTC m=+71.924197183" Sep 06 19:28:49.202934 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.202888 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p2qnt" podStartSLOduration=69.407537565 podStartE2EDuration="1m11.202875204s" totalImagesPullingTime="1.795337639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:49.202491061 +0000 UTC m=+71.937095793" watchObservedRunningTime="2026-09-06 19:28:49.202875204 +0000 UTC m=+71.937479936" Sep 06 19:28:49.217904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.217837 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-48bqg" podStartSLOduration=33.189195731 podStartE2EDuration="37.217817006s" totalImagesPullingTime="4.028621275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:49.217180532 +0000 UTC m=+71.951785267" watchObservedRunningTime="2026-09-06 19:28:49.217817006 +0000 UTC m=+71.952421787" Sep 06 19:28:49.234673 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:49.234621 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-l6499" podStartSLOduration=17.82114908 podStartE2EDuration="22.234603759s" totalImagesPullingTime="4.413454679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 19:28:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:49.233617938 +0000 UTC m=+71.968222671" watchObservedRunningTime="2026-09-06 19:28:49.234603759 +0000 UTC m=+71.969208490" Sep 06 19:28:53.185400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.185363 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" event={"ID":"39cce6d2-9a58-4bec-9c75-89e3215c09e6","Type":"ContainerStarted","Data":"f7acdea1522c49d0d9618fd16d8a8419510a46785a4778d460902f61aeb16336"} Sep 06 19:28:53.185799 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.185589 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:53.187393 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.187373 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" Sep 06 19:28:53.198591 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.198549 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7969fdb55c-n6b5r" podStartSLOduration=1.530381322 podStartE2EDuration="5.198535059s" totalImagesPullingTime="3.668153737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:53.198308781 +0000 UTC m=+75.932913525" watchObservedRunningTime="2026-09-06 19:28:53.198535059 +0000 UTC m=+75.933139791" Sep 06 19:28:53.431804 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.431768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:53.431804 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.431807 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:53.432405 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.432384 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34fe05ee-374e-477f-96b3-df13f2182b01-service-ca-bundle\") pod \"router-default-6bc94b9958-gc6br\" (UID: \"34fe05ee-374e-477f-96b3-df13f2182b01\") " pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:53.434269 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.434248 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1bf5b94-eaf1-4e88-96b2-153f7a967e2a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-x6nsc\" (UID: \"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:53.640458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.640376 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4m8dv\"" Sep 06 19:28:53.649184 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.649165 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" Sep 06 19:28:53.679291 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.679259 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gz27p\"" Sep 06 19:28:53.688140 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.688111 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:53.918889 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.918685 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-x6nsc"] Sep 06 19:28:53.921315 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:53.921267 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1bf5b94_eaf1_4e88_96b2_153f7a967e2a.slice/crio-476d50f8bc4fedbe05be2153df8e135483e0a8238c549da1ec8c7167ffc07300 WatchSource:0}: Error finding container 476d50f8bc4fedbe05be2153df8e135483e0a8238c549da1ec8c7167ffc07300: Status 404 returned error can't find the container with id 476d50f8bc4fedbe05be2153df8e135483e0a8238c549da1ec8c7167ffc07300 Sep 06 19:28:53.960440 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:53.960309 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6bc94b9958-gc6br"] Sep 06 19:28:53.962778 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:28:53.962735 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34fe05ee_374e_477f_96b3_df13f2182b01.slice/crio-28c4bfc95b7c249f4f49b3eb4c2617f271d71dd09696ca49baf70a4af176947f WatchSource:0}: Error finding container 28c4bfc95b7c249f4f49b3eb4c2617f271d71dd09696ca49baf70a4af176947f: Status 404 returned error can't find the container with id 28c4bfc95b7c249f4f49b3eb4c2617f271d71dd09696ca49baf70a4af176947f Sep 06 19:28:54.190327 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:54.190237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" event={"ID":"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a","Type":"ContainerStarted","Data":"476d50f8bc4fedbe05be2153df8e135483e0a8238c549da1ec8c7167ffc07300"} Sep 06 19:28:54.191954 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:54.191910 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6bc94b9958-gc6br" event={"ID":"34fe05ee-374e-477f-96b3-df13f2182b01","Type":"ContainerStarted","Data":"a5dee06ba0595ffdd0e2d7bf31d213c0de85dba3d61b4e26dd1368b5ac471dde"} Sep 06 19:28:54.192077 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:54.191974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6bc94b9958-gc6br" event={"ID":"34fe05ee-374e-477f-96b3-df13f2182b01","Type":"ContainerStarted","Data":"28c4bfc95b7c249f4f49b3eb4c2617f271d71dd09696ca49baf70a4af176947f"} Sep 06 19:28:54.208185 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:54.208145 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6bc94b9958-gc6br" podStartSLOduration=33.208131713 podStartE2EDuration="33.208131713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:54.207724504 +0000 UTC m=+76.942329238" watchObservedRunningTime="2026-09-06 19:28:54.208131713 +0000 UTC m=+76.942736439" Sep 06 19:28:54.688686 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:54.688649 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:54.691062 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:54.691038 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:55.197450 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:55.197413 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" event={"ID":"d1bf5b94-eaf1-4e88-96b2-153f7a967e2a","Type":"ContainerStarted","Data":"bdc7d0635050c441647b28fac63d982e0010cb4b4a54d0a937001b03e5815c5f"} Sep 06 19:28:55.197931 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:55.197741 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:55.199082 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:55.199060 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6bc94b9958-gc6br" Sep 06 19:28:55.213968 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:55.213919 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-x6nsc" podStartSLOduration=33.166742868 podStartE2EDuration="34.213904373s" totalImagesPullingTime="1.047161505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:28:55.212929426 +0000 UTC m=+77.947534157" watchObservedRunningTime="2026-09-06 19:28:55.213904373 +0000 UTC m=+77.948509106" Sep 06 19:28:59.174642 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:28:59.174525 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-48bqg" Sep 06 19:29:00.887516 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.887482 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6858786d5-dlzxw"] Sep 06 19:29:00.890624 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.890606 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:00.893984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.893951 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-69jfs\"" Sep 06 19:29:00.893984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.893976 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 06 19:29:00.894162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.894002 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 06 19:29:00.894162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.894002 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 06 19:29:00.894162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.894072 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 06 19:29:00.894162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.894081 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 06 19:29:00.894162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.894111 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 06 19:29:00.894336 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.894229 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 06 19:29:00.899561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.899540 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6858786d5-dlzxw"] Sep 06 19:29:00.987486 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.987451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-serving-cert\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:00.987670 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.987499 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-service-ca\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:00.987670 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.987523 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm6ph\" (UniqueName: \"kubernetes.io/projected/ba423109-43fd-47a8-b429-db965a37c9e9-kube-api-access-vm6ph\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:00.987670 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.987587 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-console-config\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:00.987670 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.987626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-oauth-serving-cert\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:00.987670 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:00.987655 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-oauth-config\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.088621 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.088579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-oauth-config\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.088858 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.088645 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-serving-cert\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.088858 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.088692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-service-ca\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.088858 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.088717 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm6ph\" (UniqueName: \"kubernetes.io/projected/ba423109-43fd-47a8-b429-db965a37c9e9-kube-api-access-vm6ph\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.088858 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.088791 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-console-config\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.088858 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.088830 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-oauth-serving-cert\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.089436 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.089412 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-service-ca\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.089545 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.089464 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-oauth-serving-cert\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.089827 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.089804 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-console-config\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.091230 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.091207 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-serving-cert\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.091316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.091211 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-oauth-config\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.096174 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.096149 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm6ph\" (UniqueName: \"kubernetes.io/projected/ba423109-43fd-47a8-b429-db965a37c9e9-kube-api-access-vm6ph\") pod \"console-6858786d5-dlzxw\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.200435 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.200387 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:01.474910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:01.474838 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6858786d5-dlzxw"] Sep 06 19:29:01.477637 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:01.477610 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba423109_43fd_47a8_b429_db965a37c9e9.slice/crio-1cf6c67431f49f3580b088b209672eacc78d84e448edeaf392e0ed294a093861 WatchSource:0}: Error finding container 1cf6c67431f49f3580b088b209672eacc78d84e448edeaf392e0ed294a093861: Status 404 returned error can't find the container with id 1cf6c67431f49f3580b088b209672eacc78d84e448edeaf392e0ed294a093861 Sep 06 19:29:02.226305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:02.226268 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6858786d5-dlzxw" event={"ID":"ba423109-43fd-47a8-b429-db965a37c9e9","Type":"ContainerStarted","Data":"1cf6c67431f49f3580b088b209672eacc78d84e448edeaf392e0ed294a093861"} Sep 06 19:29:05.239633 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:05.239594 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6858786d5-dlzxw" event={"ID":"ba423109-43fd-47a8-b429-db965a37c9e9","Type":"ContainerStarted","Data":"53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800"} Sep 06 19:29:05.258489 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:05.258442 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6858786d5-dlzxw" podStartSLOduration=2.5634025940000003 podStartE2EDuration="5.258427301s" totalImagesPullingTime="2.695024707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:05.257228616 +0000 UTC m=+87.991833370" watchObservedRunningTime="2026-09-06 19:29:05.258427301 +0000 UTC m=+87.993032033" Sep 06 19:29:09.523804 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.523767 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-548689bbb-w68nr"] Sep 06 19:29:09.526254 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.526238 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.533389 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.533366 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 06 19:29:09.538880 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.538846 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-548689bbb-w68nr"] Sep 06 19:29:09.655316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-serving-cert\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.655476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655323 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-service-ca\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.655476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655346 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-trusted-ca-bundle\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.655476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655398 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ssv4\" (UniqueName: \"kubernetes.io/projected/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-kube-api-access-5ssv4\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.655476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-config\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.655476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655465 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-oauth-serving-cert\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.655658 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.655541 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-oauth-config\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756063 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-oauth-config\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-serving-cert\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756096 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-service-ca\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756128 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-trusted-ca-bundle\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756145 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ssv4\" (UniqueName: \"kubernetes.io/projected/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-kube-api-access-5ssv4\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-config\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756480 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-oauth-serving-cert\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.756905 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756878 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-service-ca\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.757020 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756948 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-config\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.757020 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.756953 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-oauth-serving-cert\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.757226 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.757205 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-trusted-ca-bundle\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.758799 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.758777 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-serving-cert\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.759122 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.759104 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-oauth-config\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.763534 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.763508 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ssv4\" (UniqueName: \"kubernetes.io/projected/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-kube-api-access-5ssv4\") pod \"console-548689bbb-w68nr\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:09.835519 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:09.835433 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:10.103737 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:10.103666 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-548689bbb-w68nr"] Sep 06 19:29:10.107324 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:10.107296 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1fd113_ce24_46f2_ba3b_c02fb3c25476.slice/crio-3487ad8ae7cd98687f13eec14666c2877727bae904c209a17502f6856feada8a WatchSource:0}: Error finding container 3487ad8ae7cd98687f13eec14666c2877727bae904c209a17502f6856feada8a: Status 404 returned error can't find the container with id 3487ad8ae7cd98687f13eec14666c2877727bae904c209a17502f6856feada8a Sep 06 19:29:10.176395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:10.176370 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c65f4f9b-whcnh" Sep 06 19:29:10.257973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:10.257936 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548689bbb-w68nr" event={"ID":"ef1fd113-ce24-46f2-ba3b-c02fb3c25476","Type":"ContainerStarted","Data":"8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814"} Sep 06 19:29:10.258132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:10.257990 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548689bbb-w68nr" event={"ID":"ef1fd113-ce24-46f2-ba3b-c02fb3c25476","Type":"ContainerStarted","Data":"3487ad8ae7cd98687f13eec14666c2877727bae904c209a17502f6856feada8a"} Sep 06 19:29:10.278716 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:10.278665 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-548689bbb-w68nr" podStartSLOduration=1.2786491500000001 podStartE2EDuration="1.27864915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:10.277291158 +0000 UTC m=+93.011895892" watchObservedRunningTime="2026-09-06 19:29:10.27864915 +0000 UTC m=+93.013253883" Sep 06 19:29:11.200854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.200820 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:11.201246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.200910 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:11.205675 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.205648 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:11.265554 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.265526 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:11.349074 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.349033 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m"] Sep 06 19:29:11.351434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.351414 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9k76n"] Sep 06 19:29:11.351554 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.351535 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.353501 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.353486 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.354438 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.354418 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 06 19:29:11.354438 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.354429 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 06 19:29:11.354569 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.354435 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 06 19:29:11.354776 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.354737 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 06 19:29:11.354844 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.354829 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s85rc\"" Sep 06 19:29:11.354887 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.354831 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 06 19:29:11.355700 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.355684 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nlmzv\"" Sep 06 19:29:11.355996 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.355982 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 06 19:29:11.356064 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.355994 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 06 19:29:11.356299 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.356282 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 06 19:29:11.359350 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.359329 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g6qgw"] Sep 06 19:29:11.361702 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.361681 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.364064 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.364045 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 06 19:29:11.364425 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.364408 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8lnxb\"" Sep 06 19:29:11.364425 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.364419 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 06 19:29:11.364653 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.364638 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 06 19:29:11.370684 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.370661 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m"] Sep 06 19:29:11.372208 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.372187 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9k76n"] Sep 06 19:29:11.470273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-accelerators-collector-config\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470222 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.470273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470253 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-tls\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470269 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e38bc82-d6b2-4338-9568-a5b4a515afa1-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470335 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhcvb\" (UniqueName: \"kubernetes.io/projected/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-api-access-qhcvb\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470364 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f40225d8-c52b-45cc-9140-002f4aaa4c05-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470388 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-root\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470403 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29ddf\" (UniqueName: \"kubernetes.io/projected/71adaefa-a7e3-47a9-936a-5351a87a42be-kube-api-access-29ddf\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470488 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-sys\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470510 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-textfile\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470561 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f40225d8-c52b-45cc-9140-002f4aaa4c05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.470863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7e38bc82-d6b2-4338-9568-a5b4a515afa1-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.470863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f40225d8-c52b-45cc-9140-002f4aaa4c05-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.470863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-wtmp\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71adaefa-a7e3-47a9-936a-5351a87a42be-metrics-client-ca\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.470863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.470659 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdzms\" (UniqueName: \"kubernetes.io/projected/f40225d8-c52b-45cc-9140-002f4aaa4c05-kube-api-access-rdzms\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.571541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571509 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-tls\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.571541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e38bc82-d6b2-4338-9568-a5b4a515afa1-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571566 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhcvb\" (UniqueName: \"kubernetes.io/projected/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-api-access-qhcvb\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571584 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f40225d8-c52b-45cc-9140-002f4aaa4c05-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571606 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-root\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29ddf\" (UniqueName: \"kubernetes.io/projected/71adaefa-a7e3-47a9-936a-5351a87a42be-kube-api-access-29ddf\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:29:11.571674 2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571694 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-sys\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571722 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.571791 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:29:11.571765 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-tls podName:71adaefa-a7e3-47a9-936a-5351a87a42be nodeName:}" failed. No retries permitted until 2026-09-06 19:29:12.071731233 +0000 UTC m=+94.806335944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-tls") pod "node-exporter-g6qgw" (UID: "71adaefa-a7e3-47a9-936a-5351a87a42be") : secret "node-exporter-tls" not found Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571798 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-textfile\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571818 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f40225d8-c52b-45cc-9140-002f4aaa4c05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571864 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7e38bc82-d6b2-4338-9568-a5b4a515afa1-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f40225d8-c52b-45cc-9140-002f4aaa4c05-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571910 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-wtmp\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571931 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71adaefa-a7e3-47a9-936a-5351a87a42be-metrics-client-ca\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdzms\" (UniqueName: \"kubernetes.io/projected/f40225d8-c52b-45cc-9140-002f4aaa4c05-kube-api-access-rdzms\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-accelerators-collector-config\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.571989 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.572225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.572182 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-wtmp\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.572429 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f40225d8-c52b-45cc-9140-002f4aaa4c05-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.572734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.572479 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e38bc82-d6b2-4338-9568-a5b4a515afa1-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.572734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.572559 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-sys\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.572734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.572694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71adaefa-a7e3-47a9-936a-5351a87a42be-root\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.573400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.573077 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7e38bc82-d6b2-4338-9568-a5b4a515afa1-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.573400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.573327 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.573400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.573353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-textfile\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.573400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.573394 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-accelerators-collector-config\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.573661 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.573402 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71adaefa-a7e3-47a9-936a-5351a87a42be-metrics-client-ca\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.574908 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.574882 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.575054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.575036 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.575650 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.575626 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f40225d8-c52b-45cc-9140-002f4aaa4c05-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.575771 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.575683 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.575976 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.575958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f40225d8-c52b-45cc-9140-002f4aaa4c05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.585451 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.585378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29ddf\" (UniqueName: \"kubernetes.io/projected/71adaefa-a7e3-47a9-936a-5351a87a42be-kube-api-access-29ddf\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:11.585588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.585563 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhcvb\" (UniqueName: \"kubernetes.io/projected/7e38bc82-d6b2-4338-9568-a5b4a515afa1-kube-api-access-qhcvb\") pod \"kube-state-metrics-74cb57df7-9k76n\" (UID: \"7e38bc82-d6b2-4338-9568-a5b4a515afa1\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.586117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.586097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdzms\" (UniqueName: \"kubernetes.io/projected/f40225d8-c52b-45cc-9140-002f4aaa4c05-kube-api-access-rdzms\") pod \"openshift-state-metrics-657fd6c54-zh44m\" (UID: \"f40225d8-c52b-45cc-9140-002f4aaa4c05\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.663088 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.663061 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" Sep 06 19:29:11.669962 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.669940 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" Sep 06 19:29:11.939206 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.938975 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m"] Sep 06 19:29:11.943321 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:11.943296 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40225d8_c52b_45cc_9140_002f4aaa4c05.slice/crio-a9839667365563ed52fbd1b480b9a936f6009b1056b6a5f3789e72359e2fc324 WatchSource:0}: Error finding container a9839667365563ed52fbd1b480b9a936f6009b1056b6a5f3789e72359e2fc324: Status 404 returned error can't find the container with id a9839667365563ed52fbd1b480b9a936f6009b1056b6a5f3789e72359e2fc324 Sep 06 19:29:11.951386 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:11.951360 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9k76n"] Sep 06 19:29:11.957593 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:11.957566 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e38bc82_d6b2_4338_9568_a5b4a515afa1.slice/crio-49ff08c0fa8b9c7e83210cda04df8888e77ba6fa0076d5ffd6ee51a3ea022478 WatchSource:0}: Error finding container 49ff08c0fa8b9c7e83210cda04df8888e77ba6fa0076d5ffd6ee51a3ea022478: Status 404 returned error can't find the container with id 49ff08c0fa8b9c7e83210cda04df8888e77ba6fa0076d5ffd6ee51a3ea022478 Sep 06 19:29:12.075976 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.075940 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-tls\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:12.078243 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.078208 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71adaefa-a7e3-47a9-936a-5351a87a42be-node-exporter-tls\") pod \"node-exporter-g6qgw\" (UID: \"71adaefa-a7e3-47a9-936a-5351a87a42be\") " pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:12.272220 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.272126 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" event={"ID":"7e38bc82-d6b2-4338-9568-a5b4a515afa1","Type":"ContainerStarted","Data":"49ff08c0fa8b9c7e83210cda04df8888e77ba6fa0076d5ffd6ee51a3ea022478"} Sep 06 19:29:12.274404 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.274375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" event={"ID":"f40225d8-c52b-45cc-9140-002f4aaa4c05","Type":"ContainerStarted","Data":"57f86293affbd3c288b9b387f4bc9d348c2ccf059df8af270d78edd9b71205de"} Sep 06 19:29:12.274542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.274426 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" event={"ID":"f40225d8-c52b-45cc-9140-002f4aaa4c05","Type":"ContainerStarted","Data":"a65d8b4d4cb6f965caa007f9f76a70f57c84a219576fd41627644fd777053a0f"} Sep 06 19:29:12.274542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.274445 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" event={"ID":"f40225d8-c52b-45cc-9140-002f4aaa4c05","Type":"ContainerStarted","Data":"a9839667365563ed52fbd1b480b9a936f6009b1056b6a5f3789e72359e2fc324"} Sep 06 19:29:12.275692 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:12.275675 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g6qgw" Sep 06 19:29:12.285237 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:12.285216 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71adaefa_a7e3_47a9_936a_5351a87a42be.slice/crio-c4303a1e7059e7e3ded781255b556ca6f9dd7d5a336a3d0892be09fff8927b6d WatchSource:0}: Error finding container c4303a1e7059e7e3ded781255b556ca6f9dd7d5a336a3d0892be09fff8927b6d: Status 404 returned error can't find the container with id c4303a1e7059e7e3ded781255b556ca6f9dd7d5a336a3d0892be09fff8927b6d Sep 06 19:29:13.281005 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:13.280960 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g6qgw" event={"ID":"71adaefa-a7e3-47a9-936a-5351a87a42be","Type":"ContainerStarted","Data":"c4303a1e7059e7e3ded781255b556ca6f9dd7d5a336a3d0892be09fff8927b6d"} Sep 06 19:29:14.287315 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.287281 2575 generic.go:356] "Generic (PLEG): container finished" podID="71adaefa-a7e3-47a9-936a-5351a87a42be" containerID="bc94549e3cfb0e8dba34ffeb81e9db740a80098c65c50fe805670b48a53c60ed" exitCode=0 Sep 06 19:29:14.287861 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.287372 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g6qgw" event={"ID":"71adaefa-a7e3-47a9-936a-5351a87a42be","Type":"ContainerDied","Data":"bc94549e3cfb0e8dba34ffeb81e9db740a80098c65c50fe805670b48a53c60ed"} Sep 06 19:29:14.289627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.289590 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" event={"ID":"7e38bc82-d6b2-4338-9568-a5b4a515afa1","Type":"ContainerStarted","Data":"3a83cb2f29add0ffcf1392ade07d918a3e31b49b1e7b60b1db337ed0c2fbe127"} Sep 06 19:29:14.289770 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.289663 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" event={"ID":"7e38bc82-d6b2-4338-9568-a5b4a515afa1","Type":"ContainerStarted","Data":"ae080bbc6119a1b41dce7e982d571f27d5b79e925f8e420334beef4ea7441138"} Sep 06 19:29:14.289770 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.289690 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" event={"ID":"7e38bc82-d6b2-4338-9568-a5b4a515afa1","Type":"ContainerStarted","Data":"c415a57ebd17fa487dfe198fedc8b17730bf1d565af98e7428bc4849788f703c"} Sep 06 19:29:14.291948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.291926 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" event={"ID":"f40225d8-c52b-45cc-9140-002f4aaa4c05","Type":"ContainerStarted","Data":"4b7d12265b9d13300ad7f055f3649a7cca25677c21af80a86c57cab790a65e51"} Sep 06 19:29:14.326107 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.326054 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zh44m" podStartSLOduration=1.799266901 podStartE2EDuration="3.326032887s" totalImagesPullingTime="1.526765986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:14.325712053 +0000 UTC m=+97.060316786" watchObservedRunningTime="2026-09-06 19:29:14.326032887 +0000 UTC m=+97.060637622" Sep 06 19:29:14.345595 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:14.345530 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9k76n" podStartSLOduration=1.7069711 podStartE2EDuration="3.345511613s" totalImagesPullingTime="1.638540513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:14.344939025 +0000 UTC m=+97.079543758" watchObservedRunningTime="2026-09-06 19:29:14.345511613 +0000 UTC m=+97.080116347" Sep 06 19:29:15.297632 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:15.297594 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g6qgw" event={"ID":"71adaefa-a7e3-47a9-936a-5351a87a42be","Type":"ContainerStarted","Data":"3c981bcf2f145161e62898f4175807d5eca56cd42822440b3c4bab2437e222a7"} Sep 06 19:29:15.298098 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:15.297677 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g6qgw" event={"ID":"71adaefa-a7e3-47a9-936a-5351a87a42be","Type":"ContainerStarted","Data":"1dafdaa37f61d68d47f7539cbcb6e9669dcbec47cf448c5ae6a8a18251163e79"} Sep 06 19:29:15.315899 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:15.315844 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g6qgw" podStartSLOduration=2.902346839 podStartE2EDuration="4.315828926s" totalImagesPullingTime="1.313087246s" totalInitContainerRuntime="100.394841ms" isStatefulPod=true podCreationTimestamp="2026-09-06 19:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:15.31397367 +0000 UTC m=+98.048578395" watchObservedRunningTime="2026-09-06 19:29:15.315828926 +0000 UTC m=+98.050433658" Sep 06 19:29:16.110703 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.110624 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b"] Sep 06 19:29:16.116376 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.116358 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.118714 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.118683 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 06 19:29:16.118840 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.118696 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pd2vj\"" Sep 06 19:29:16.122949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.122928 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b"] Sep 06 19:29:16.143277 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.143250 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9j4ml" Sep 06 19:29:16.146263 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.146238 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-548689bbb-w68nr"] Sep 06 19:29:16.179787 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.179715 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bd65bcd87-jgds8"] Sep 06 19:29:16.183940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.183923 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.190648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.190620 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bd65bcd87-jgds8"] Sep 06 19:29:16.208984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.208952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.209137 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.209082 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnj4d\" (UniqueName: \"kubernetes.io/projected/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-kube-api-access-rnj4d\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.309628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnj4d\" (UniqueName: \"kubernetes.io/projected/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-kube-api-access-rnj4d\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309645 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7cv2\" (UniqueName: \"kubernetes.io/projected/0eab4929-eac5-421e-bf6b-4f5b52ec037c-kube-api-access-s7cv2\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309673 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-service-ca\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309719 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309762 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-serving-cert\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309790 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-config\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309819 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-oauth-serving-cert\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-trusted-ca-bundle\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:29:16.309858 2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:29:16.309912 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-monitoring-plugin-cert podName:73c87f71-a35f-4bcc-b91e-9aaa6576e33b nodeName:}" failed. No retries permitted until 2026-09-06 19:29:16.809896132 +0000 UTC m=+99.544500841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-6688b" (UID: "73c87f71-a35f-4bcc-b91e-9aaa6576e33b") : secret "monitoring-plugin-cert" not found Sep 06 19:29:16.310087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.309938 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-oauth-config\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.324156 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.324118 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnj4d\" (UniqueName: \"kubernetes.io/projected/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-kube-api-access-rnj4d\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.411375 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411259 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-trusted-ca-bundle\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.411375 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-oauth-config\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.411600 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411463 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7cv2\" (UniqueName: \"kubernetes.io/projected/0eab4929-eac5-421e-bf6b-4f5b52ec037c-kube-api-access-s7cv2\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.411600 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-service-ca\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.411705 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-serving-cert\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.411705 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-config\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.411837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.411718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-oauth-serving-cert\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.412375 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.412332 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-trusted-ca-bundle\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.412643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.412372 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-config\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.412643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.412469 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-oauth-serving-cert\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.412850 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.412829 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-service-ca\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.414171 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.414151 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-oauth-config\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.414428 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.414410 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-serving-cert\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.418878 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.418860 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7cv2\" (UniqueName: \"kubernetes.io/projected/0eab4929-eac5-421e-bf6b-4f5b52ec037c-kube-api-access-s7cv2\") pod \"console-6bd65bcd87-jgds8\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.494673 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.494629 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:16.770538 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.770503 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bd65bcd87-jgds8"] Sep 06 19:29:16.773996 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:16.773965 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eab4929_eac5_421e_bf6b_4f5b52ec037c.slice/crio-346b87909a8d5dc0c3e618069dd1e0301fea087f2bd7c31e4b8a8c0a568fc5d9 WatchSource:0}: Error finding container 346b87909a8d5dc0c3e618069dd1e0301fea087f2bd7c31e4b8a8c0a568fc5d9: Status 404 returned error can't find the container with id 346b87909a8d5dc0c3e618069dd1e0301fea087f2bd7c31e4b8a8c0a568fc5d9 Sep 06 19:29:16.814729 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.814702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:16.817332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:16.817304 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73c87f71-a35f-4bcc-b91e-9aaa6576e33b-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-6688b\" (UID: \"73c87f71-a35f-4bcc-b91e-9aaa6576e33b\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:17.026564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:17.026470 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:17.294686 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:17.294596 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b"] Sep 06 19:29:17.297340 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:29:17.297302 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c87f71_a35f_4bcc_b91e_9aaa6576e33b.slice/crio-c35b13a1219b36f5ee2f0ee9c1544ca9866d200d338671a0517f855c0dcf68f2 WatchSource:0}: Error finding container c35b13a1219b36f5ee2f0ee9c1544ca9866d200d338671a0517f855c0dcf68f2: Status 404 returned error can't find the container with id c35b13a1219b36f5ee2f0ee9c1544ca9866d200d338671a0517f855c0dcf68f2 Sep 06 19:29:17.309153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:17.309123 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd65bcd87-jgds8" event={"ID":"0eab4929-eac5-421e-bf6b-4f5b52ec037c","Type":"ContainerStarted","Data":"8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e"} Sep 06 19:29:17.309258 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:17.309177 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd65bcd87-jgds8" event={"ID":"0eab4929-eac5-421e-bf6b-4f5b52ec037c","Type":"ContainerStarted","Data":"346b87909a8d5dc0c3e618069dd1e0301fea087f2bd7c31e4b8a8c0a568fc5d9"} Sep 06 19:29:17.310191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:17.310170 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" event={"ID":"73c87f71-a35f-4bcc-b91e-9aaa6576e33b","Type":"ContainerStarted","Data":"c35b13a1219b36f5ee2f0ee9c1544ca9866d200d338671a0517f855c0dcf68f2"} Sep 06 19:29:17.329130 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:17.329072 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bd65bcd87-jgds8" podStartSLOduration=1.329054777 podStartE2EDuration="1.329054777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:17.328451789 +0000 UTC m=+100.063056518" watchObservedRunningTime="2026-09-06 19:29:17.329054777 +0000 UTC m=+100.063659510" Sep 06 19:29:19.321129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:19.321090 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" event={"ID":"73c87f71-a35f-4bcc-b91e-9aaa6576e33b","Type":"ContainerStarted","Data":"78b7d4f2071805965f0a81c67d7523256041eb12808547de8c7716875af4306a"} Sep 06 19:29:19.321521 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:19.321250 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:19.325809 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:19.325786 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" Sep 06 19:29:19.335936 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:19.335897 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-6688b" podStartSLOduration=2.01155297 podStartE2EDuration="3.335883182s" totalImagesPullingTime="1.324330212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:29:19.334906807 +0000 UTC m=+102.069511540" watchObservedRunningTime="2026-09-06 19:29:19.335883182 +0000 UTC m=+102.070487915" Sep 06 19:29:19.835937 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:19.835889 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:26.495467 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:26.495433 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:26.495893 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:26.495805 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:26.500372 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:26.500351 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:27.358515 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:27.358488 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:29:27.399160 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:27.399124 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6858786d5-dlzxw"] Sep 06 19:29:41.167428 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.167361 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-548689bbb-w68nr" podUID="ef1fd113-ce24-46f2-ba3b-c02fb3c25476" containerName="console" containerID="cri-o://8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814" gracePeriod=15 Sep 06 19:29:41.406802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.406777 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:41.413984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.413957 2575 generic.go:356] "Generic (PLEG): container finished" podID="ef1fd113-ce24-46f2-ba3b-c02fb3c25476" containerID="8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814" exitCode=0 Sep 06 19:29:41.414105 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.413995 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548689bbb-w68nr" Sep 06 19:29:41.414105 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.414018 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548689bbb-w68nr" event={"ID":"ef1fd113-ce24-46f2-ba3b-c02fb3c25476","Type":"ContainerDied","Data":"8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814"} Sep 06 19:29:41.414105 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.414069 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548689bbb-w68nr" event={"ID":"ef1fd113-ce24-46f2-ba3b-c02fb3c25476","Type":"ContainerDied","Data":"3487ad8ae7cd98687f13eec14666c2877727bae904c209a17502f6856feada8a"} Sep 06 19:29:41.414105 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.414096 2575 scope.go:117] "RemoveContainer" containerID="8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814" Sep 06 19:29:41.421373 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421309 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-config\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.421373 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421356 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-oauth-serving-cert\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.421520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421408 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-oauth-config\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.421520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421428 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ssv4\" (UniqueName: \"kubernetes.io/projected/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-kube-api-access-5ssv4\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.421520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421445 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-trusted-ca-bundle\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.421520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421487 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-service-ca\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.421704 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421514 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-serving-cert\") pod \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\" (UID: \"ef1fd113-ce24-46f2-ba3b-c02fb3c25476\") " Sep 06 19:29:41.422365 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.421800 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-config" (OuterVolumeSpecName: "console-config") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:41.422365 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.422007 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.423185 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.422972 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-service-ca" (OuterVolumeSpecName: "service-ca") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:41.423185 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.423046 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:41.423185 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.423035 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:41.423395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.423291 2575 scope.go:117] "RemoveContainer" containerID="8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814" Sep 06 19:29:41.423648 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:29:41.423625 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814\": container with ID starting with 8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814 not found: ID does not exist" containerID="8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814" Sep 06 19:29:41.423730 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.423654 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814"} err="failed to get container status \"8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814\": rpc error: code = NotFound desc = could not find container \"8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814\": container with ID starting with 8391d36f2bd8796753664f12fc6d4dac36fd139831efc1b5cfa14c844c3cb814 not found: ID does not exist" Sep 06 19:29:41.425072 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.424997 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:29:41.425175 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.425080 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-kube-api-access-5ssv4" (OuterVolumeSpecName: "kube-api-access-5ssv4") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "kube-api-access-5ssv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:29:41.425528 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.425501 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ef1fd113-ce24-46f2-ba3b-c02fb3c25476" (UID: "ef1fd113-ce24-46f2-ba3b-c02fb3c25476"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:29:41.523117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.523074 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-oauth-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.523117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.523104 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-oauth-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.523117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.523114 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ssv4\" (UniqueName: \"kubernetes.io/projected/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-kube-api-access-5ssv4\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.523117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.523125 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-trusted-ca-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.523353 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.523134 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.523353 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.523143 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef1fd113-ce24-46f2-ba3b-c02fb3c25476-console-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:41.733953 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.733928 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-548689bbb-w68nr"] Sep 06 19:29:41.737589 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.737564 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-548689bbb-w68nr"] Sep 06 19:29:41.764800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:41.762416 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef1fd113-ce24-46f2-ba3b-c02fb3c25476" path="/var/lib/kubelet/pods/ef1fd113-ce24-46f2-ba3b-c02fb3c25476/volumes" Sep 06 19:29:42.420135 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:42.420102 2575 generic.go:356] "Generic (PLEG): container finished" podID="d15c8c28-2067-439e-b876-0575479656de" containerID="37557ff79910ae35b07a5ee6850a6e126c995186a9d996d0e12365a2c58f9b38" exitCode=0 Sep 06 19:29:42.420562 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:42.420169 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" event={"ID":"d15c8c28-2067-439e-b876-0575479656de","Type":"ContainerDied","Data":"37557ff79910ae35b07a5ee6850a6e126c995186a9d996d0e12365a2c58f9b38"} Sep 06 19:29:42.420562 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:42.420553 2575 scope.go:117] "RemoveContainer" containerID="37557ff79910ae35b07a5ee6850a6e126c995186a9d996d0e12365a2c58f9b38" Sep 06 19:29:42.421947 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:42.421849 2575 generic.go:356] "Generic (PLEG): container finished" podID="09828082-8a90-466f-85ff-02776367a618" containerID="a0a53d406f1347148e694321cdada26927bc0cc3cc4954ffdfbe172f9929f06e" exitCode=0 Sep 06 19:29:42.421947 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:42.421889 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" event={"ID":"09828082-8a90-466f-85ff-02776367a618","Type":"ContainerDied","Data":"a0a53d406f1347148e694321cdada26927bc0cc3cc4954ffdfbe172f9929f06e"} Sep 06 19:29:42.422148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:42.422133 2575 scope.go:117] "RemoveContainer" containerID="a0a53d406f1347148e694321cdada26927bc0cc3cc4954ffdfbe172f9929f06e" Sep 06 19:29:43.427921 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:43.427884 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dblnk" event={"ID":"d15c8c28-2067-439e-b876-0575479656de","Type":"ContainerStarted","Data":"ca765698ca74771c3bd7be530e83b1927e65e8da68e37fd196ecbf12bf4e9eef"} Sep 06 19:29:43.429642 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:43.429618 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-d7hpj" event={"ID":"09828082-8a90-466f-85ff-02776367a618","Type":"ContainerStarted","Data":"421f5c844c250b903dae4d3044212f192dc9d8b38b9a78d160177a8289a25a77"} Sep 06 19:29:52.420829 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.420779 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6858786d5-dlzxw" podUID="ba423109-43fd-47a8-b429-db965a37c9e9" containerName="console" containerID="cri-o://53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800" gracePeriod=15 Sep 06 19:29:52.667668 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.667645 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:52.721255 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721171 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-oauth-serving-cert\") pod \"ba423109-43fd-47a8-b429-db965a37c9e9\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " Sep 06 19:29:52.721255 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721215 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-console-config\") pod \"ba423109-43fd-47a8-b429-db965a37c9e9\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " Sep 06 19:29:52.721491 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721283 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-oauth-config\") pod \"ba423109-43fd-47a8-b429-db965a37c9e9\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " Sep 06 19:29:52.721491 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721306 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm6ph\" (UniqueName: \"kubernetes.io/projected/ba423109-43fd-47a8-b429-db965a37c9e9-kube-api-access-vm6ph\") pod \"ba423109-43fd-47a8-b429-db965a37c9e9\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " Sep 06 19:29:52.721491 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721326 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-service-ca\") pod \"ba423109-43fd-47a8-b429-db965a37c9e9\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " Sep 06 19:29:52.721491 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721347 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-serving-cert\") pod \"ba423109-43fd-47a8-b429-db965a37c9e9\" (UID: \"ba423109-43fd-47a8-b429-db965a37c9e9\") " Sep 06 19:29:52.721691 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721654 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ba423109-43fd-47a8-b429-db965a37c9e9" (UID: "ba423109-43fd-47a8-b429-db965a37c9e9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:52.721784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721701 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-service-ca" (OuterVolumeSpecName: "service-ca") pod "ba423109-43fd-47a8-b429-db965a37c9e9" (UID: "ba423109-43fd-47a8-b429-db965a37c9e9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:52.721784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.721717 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-console-config" (OuterVolumeSpecName: "console-config") pod "ba423109-43fd-47a8-b429-db965a37c9e9" (UID: "ba423109-43fd-47a8-b429-db965a37c9e9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:29:52.723728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.723699 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba423109-43fd-47a8-b429-db965a37c9e9-kube-api-access-vm6ph" (OuterVolumeSpecName: "kube-api-access-vm6ph") pod "ba423109-43fd-47a8-b429-db965a37c9e9" (UID: "ba423109-43fd-47a8-b429-db965a37c9e9"). InnerVolumeSpecName "kube-api-access-vm6ph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:29:52.723837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.723733 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ba423109-43fd-47a8-b429-db965a37c9e9" (UID: "ba423109-43fd-47a8-b429-db965a37c9e9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:29:52.723837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.723787 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ba423109-43fd-47a8-b429-db965a37c9e9" (UID: "ba423109-43fd-47a8-b429-db965a37c9e9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:29:52.822086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.822051 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-oauth-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:52.822086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.822078 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm6ph\" (UniqueName: \"kubernetes.io/projected/ba423109-43fd-47a8-b429-db965a37c9e9-kube-api-access-vm6ph\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:52.822086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.822089 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:52.822086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.822098 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba423109-43fd-47a8-b429-db965a37c9e9-console-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:52.822339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.822106 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-oauth-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:52.822339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:52.822116 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba423109-43fd-47a8-b429-db965a37c9e9-console-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:29:53.470740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.470709 2575 generic.go:356] "Generic (PLEG): container finished" podID="ba423109-43fd-47a8-b429-db965a37c9e9" containerID="53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800" exitCode=0 Sep 06 19:29:53.471154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.470801 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6858786d5-dlzxw" Sep 06 19:29:53.471154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.470794 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6858786d5-dlzxw" event={"ID":"ba423109-43fd-47a8-b429-db965a37c9e9","Type":"ContainerDied","Data":"53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800"} Sep 06 19:29:53.471154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.470929 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6858786d5-dlzxw" event={"ID":"ba423109-43fd-47a8-b429-db965a37c9e9","Type":"ContainerDied","Data":"1cf6c67431f49f3580b088b209672eacc78d84e448edeaf392e0ed294a093861"} Sep 06 19:29:53.471154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.470950 2575 scope.go:117] "RemoveContainer" containerID="53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800" Sep 06 19:29:53.480181 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.480073 2575 scope.go:117] "RemoveContainer" containerID="53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800" Sep 06 19:29:53.480383 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:29:53.480356 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800\": container with ID starting with 53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800 not found: ID does not exist" containerID="53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800" Sep 06 19:29:53.480500 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.480388 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800"} err="failed to get container status \"53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800\": rpc error: code = NotFound desc = could not find container \"53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800\": container with ID starting with 53b890d3a37815938e9597c5181945d75c2493cbbde5996782e88efae8e2b800 not found: ID does not exist" Sep 06 19:29:53.491821 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.491766 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6858786d5-dlzxw"] Sep 06 19:29:53.495353 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.495327 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6858786d5-dlzxw"] Sep 06 19:29:53.761616 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:29:53.761541 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba423109-43fd-47a8-b429-db965a37c9e9" path="/var/lib/kubelet/pods/ba423109-43fd-47a8-b429-db965a37c9e9/volumes" Sep 06 19:30:38.939032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.938997 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77d6c95b8f-9qkjn"] Sep 06 19:30:38.939437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.939368 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef1fd113-ce24-46f2-ba3b-c02fb3c25476" containerName="console" Sep 06 19:30:38.939437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.939385 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef1fd113-ce24-46f2-ba3b-c02fb3c25476" containerName="console" Sep 06 19:30:38.939437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.939396 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba423109-43fd-47a8-b429-db965a37c9e9" containerName="console" Sep 06 19:30:38.939437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.939401 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba423109-43fd-47a8-b429-db965a37c9e9" containerName="console" Sep 06 19:30:38.939588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.939460 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba423109-43fd-47a8-b429-db965a37c9e9" containerName="console" Sep 06 19:30:38.939588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.939468 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef1fd113-ce24-46f2-ba3b-c02fb3c25476" containerName="console" Sep 06 19:30:38.943548 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.943532 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.953285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.953255 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77d6c95b8f-9qkjn"] Sep 06 19:30:38.978837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.978787 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-oauth-serving-cert\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.978837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.978839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-console-config\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.979032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.978922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldjs9\" (UniqueName: \"kubernetes.io/projected/66157c67-a610-4adc-9887-442858073a01-kube-api-access-ldjs9\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.979032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.978956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-oauth-config\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.979032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.978994 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-trusted-ca-bundle\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.979130 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.979055 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-serving-cert\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:38.979130 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:38.979069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-service-ca\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079450 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-oauth-config\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079462 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-trusted-ca-bundle\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-serving-cert\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-service-ca\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-oauth-serving-cert\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-console-config\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.079618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.079561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldjs9\" (UniqueName: \"kubernetes.io/projected/66157c67-a610-4adc-9887-442858073a01-kube-api-access-ldjs9\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.080494 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.080465 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-oauth-serving-cert\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.080732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.080710 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-service-ca\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.080828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.080733 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-console-config\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.080893 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.080850 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-trusted-ca-bundle\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.081983 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.081962 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-oauth-config\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.082101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.082085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-serving-cert\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.086845 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.086825 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldjs9\" (UniqueName: \"kubernetes.io/projected/66157c67-a610-4adc-9887-442858073a01-kube-api-access-ldjs9\") pod \"console-77d6c95b8f-9qkjn\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.253818 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.253694 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:39.523861 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.523828 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77d6c95b8f-9qkjn"] Sep 06 19:30:39.526242 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:30:39.526212 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66157c67_a610_4adc_9887_442858073a01.slice/crio-898043356e8badf91181193c3ee29de61c1070b78bdc6e1b451ab0911bdeecca WatchSource:0}: Error finding container 898043356e8badf91181193c3ee29de61c1070b78bdc6e1b451ab0911bdeecca: Status 404 returned error can't find the container with id 898043356e8badf91181193c3ee29de61c1070b78bdc6e1b451ab0911bdeecca Sep 06 19:30:39.649657 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.649628 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d6c95b8f-9qkjn" event={"ID":"66157c67-a610-4adc-9887-442858073a01","Type":"ContainerStarted","Data":"5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b"} Sep 06 19:30:39.649883 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.649685 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d6c95b8f-9qkjn" event={"ID":"66157c67-a610-4adc-9887-442858073a01","Type":"ContainerStarted","Data":"898043356e8badf91181193c3ee29de61c1070b78bdc6e1b451ab0911bdeecca"} Sep 06 19:30:39.675772 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:39.668103 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77d6c95b8f-9qkjn" podStartSLOduration=1.668079705 podStartE2EDuration="1.668079705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:30:39.667029344 +0000 UTC m=+182.401634077" watchObservedRunningTime="2026-09-06 19:30:39.668079705 +0000 UTC m=+182.402684440" Sep 06 19:30:49.254655 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:49.254616 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:49.255093 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:49.254736 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:49.259051 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:49.259028 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:49.691802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:49.691774 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:30:49.724431 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:30:49.724399 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bd65bcd87-jgds8"] Sep 06 19:31:14.746949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:14.746887 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bd65bcd87-jgds8" podUID="0eab4929-eac5-421e-bf6b-4f5b52ec037c" containerName="console" containerID="cri-o://8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e" gracePeriod=15 Sep 06 19:31:14.999494 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:14.999435 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:31:15.068981 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.068946 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-config\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069152 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069000 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-oauth-serving-cert\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069152 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069026 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-service-ca\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069152 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069067 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-trusted-ca-bundle\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069152 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069098 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-oauth-config\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069153 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7cv2\" (UniqueName: \"kubernetes.io/projected/0eab4929-eac5-421e-bf6b-4f5b52ec037c-kube-api-access-s7cv2\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069185 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-serving-cert\") pod \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\" (UID: \"0eab4929-eac5-421e-bf6b-4f5b52ec037c\") " Sep 06 19:31:15.069363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069303 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-config" (OuterVolumeSpecName: "console-config") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:31:15.069520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069431 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:31:15.069520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069508 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-service-ca" (OuterVolumeSpecName: "service-ca") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:31:15.069603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069552 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:31:15.069603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069598 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-oauth-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.069669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069612 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.069669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.069620 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.071547 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.071499 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eab4929-eac5-421e-bf6b-4f5b52ec037c-kube-api-access-s7cv2" (OuterVolumeSpecName: "kube-api-access-s7cv2") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "kube-api-access-s7cv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:31:15.071547 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.071533 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:31:15.071681 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.071559 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0eab4929-eac5-421e-bf6b-4f5b52ec037c" (UID: "0eab4929-eac5-421e-bf6b-4f5b52ec037c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:31:15.170242 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.170209 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eab4929-eac5-421e-bf6b-4f5b52ec037c-trusted-ca-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.170242 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.170236 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-oauth-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.170242 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.170245 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7cv2\" (UniqueName: \"kubernetes.io/projected/0eab4929-eac5-421e-bf6b-4f5b52ec037c-kube-api-access-s7cv2\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.170463 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.170254 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eab4929-eac5-421e-bf6b-4f5b52ec037c-console-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:31:15.783834 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.783805 2575 generic.go:356] "Generic (PLEG): container finished" podID="0eab4929-eac5-421e-bf6b-4f5b52ec037c" containerID="8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e" exitCode=0 Sep 06 19:31:15.784193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.783851 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd65bcd87-jgds8" Sep 06 19:31:15.784193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.783879 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd65bcd87-jgds8" event={"ID":"0eab4929-eac5-421e-bf6b-4f5b52ec037c","Type":"ContainerDied","Data":"8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e"} Sep 06 19:31:15.784193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.783921 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd65bcd87-jgds8" event={"ID":"0eab4929-eac5-421e-bf6b-4f5b52ec037c","Type":"ContainerDied","Data":"346b87909a8d5dc0c3e618069dd1e0301fea087f2bd7c31e4b8a8c0a568fc5d9"} Sep 06 19:31:15.784193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.783940 2575 scope.go:117] "RemoveContainer" containerID="8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e" Sep 06 19:31:15.791984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.791967 2575 scope.go:117] "RemoveContainer" containerID="8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e" Sep 06 19:31:15.792286 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:31:15.792262 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e\": container with ID starting with 8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e not found: ID does not exist" containerID="8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e" Sep 06 19:31:15.792339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.792293 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e"} err="failed to get container status \"8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e\": rpc error: code = NotFound desc = could not find container \"8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e\": container with ID starting with 8af4cb375ed490054620e3b8ad6a6e521aec5d8101dcf3a0bf886ca0a634ee9e not found: ID does not exist" Sep 06 19:31:15.803830 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.799314 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bd65bcd87-jgds8"] Sep 06 19:31:15.805742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:15.805720 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bd65bcd87-jgds8"] Sep 06 19:31:17.762111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:17.762075 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eab4929-eac5-421e-bf6b-4f5b52ec037c" path="/var/lib/kubelet/pods/0eab4929-eac5-421e-bf6b-4f5b52ec037c/volumes" Sep 06 19:31:51.219736 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.219704 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7995f84697-sj4dh"] Sep 06 19:31:51.220193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.220001 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eab4929-eac5-421e-bf6b-4f5b52ec037c" containerName="console" Sep 06 19:31:51.220193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.220012 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eab4929-eac5-421e-bf6b-4f5b52ec037c" containerName="console" Sep 06 19:31:51.220193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.220072 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eab4929-eac5-421e-bf6b-4f5b52ec037c" containerName="console" Sep 06 19:31:51.223035 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.223012 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.234005 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.233984 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7995f84697-sj4dh"] Sep 06 19:31:51.256040 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256018 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-config\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.256181 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256046 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-oauth-config\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.256181 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256080 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668ll\" (UniqueName: \"kubernetes.io/projected/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-kube-api-access-668ll\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.256181 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-serving-cert\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.256301 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-trusted-ca-bundle\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.256301 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-oauth-serving-cert\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.256301 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.256223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-service-ca\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357076 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357041 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-config\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357076 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-oauth-config\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357105 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668ll\" (UniqueName: \"kubernetes.io/projected/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-kube-api-access-668ll\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357136 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-serving-cert\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-trusted-ca-bundle\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357189 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-oauth-serving-cert\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-service-ca\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.357915 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357886 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-config\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.358025 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357939 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-oauth-serving-cert\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.358105 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.357952 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-service-ca\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.358148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.358025 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-trusted-ca-bundle\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.359719 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.359696 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-serving-cert\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.359827 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.359718 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-oauth-config\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.364267 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.364250 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-668ll\" (UniqueName: \"kubernetes.io/projected/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-kube-api-access-668ll\") pod \"console-7995f84697-sj4dh\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.532961 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.532877 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:31:51.802463 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.802374 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7995f84697-sj4dh"] Sep 06 19:31:51.805119 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:31:51.805091 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde3677f_e3f7_48c8_8f55_1d53a9581e9d.slice/crio-f5002ea0d0c9f85559619967335c1c471e456b601a4ea450065462f35e9bfaf3 WatchSource:0}: Error finding container f5002ea0d0c9f85559619967335c1c471e456b601a4ea450065462f35e9bfaf3: Status 404 returned error can't find the container with id f5002ea0d0c9f85559619967335c1c471e456b601a4ea450065462f35e9bfaf3 Sep 06 19:31:51.920061 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.920030 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7995f84697-sj4dh" event={"ID":"dde3677f-e3f7-48c8-8f55-1d53a9581e9d","Type":"ContainerStarted","Data":"749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2"} Sep 06 19:31:51.920220 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.920076 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7995f84697-sj4dh" event={"ID":"dde3677f-e3f7-48c8-8f55-1d53a9581e9d","Type":"ContainerStarted","Data":"f5002ea0d0c9f85559619967335c1c471e456b601a4ea450065462f35e9bfaf3"} Sep 06 19:31:51.935427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:31:51.935353 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7995f84697-sj4dh" podStartSLOduration=0.935334336 podStartE2EDuration="935.334336ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:31:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:31:51.934550294 +0000 UTC m=+254.669155026" watchObservedRunningTime="2026-09-06 19:31:51.935334336 +0000 UTC m=+254.669939068" Sep 06 19:32:01.533780 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:01.533674 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:32:01.533780 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:01.533731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:32:01.538722 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:01.538699 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:32:01.960310 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:01.960280 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:32:02.005380 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:02.005345 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77d6c95b8f-9qkjn"] Sep 06 19:32:27.028534 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.028493 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77d6c95b8f-9qkjn" podUID="66157c67-a610-4adc-9887-442858073a01" containerName="console" containerID="cri-o://5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b" gracePeriod=15 Sep 06 19:32:27.273740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.273712 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:32:27.345859 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345767 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-oauth-config\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.345859 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345825 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-service-ca\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.346073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345873 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-trusted-ca-bundle\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.346073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345897 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldjs9\" (UniqueName: \"kubernetes.io/projected/66157c67-a610-4adc-9887-442858073a01-kube-api-access-ldjs9\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.346073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345934 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-console-config\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.346073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345964 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-serving-cert\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.346073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.345987 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-oauth-serving-cert\") pod \"66157c67-a610-4adc-9887-442858073a01\" (UID: \"66157c67-a610-4adc-9887-442858073a01\") " Sep 06 19:32:27.346394 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.346316 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-service-ca" (OuterVolumeSpecName: "service-ca") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:32:27.346696 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.346341 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-console-config" (OuterVolumeSpecName: "console-config") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:32:27.346696 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.346440 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:32:27.346696 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.346489 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:32:27.348139 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.348111 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:32:27.348248 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.348200 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:32:27.348248 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.348207 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66157c67-a610-4adc-9887-442858073a01-kube-api-access-ldjs9" (OuterVolumeSpecName: "kube-api-access-ldjs9") pod "66157c67-a610-4adc-9887-442858073a01" (UID: "66157c67-a610-4adc-9887-442858073a01"). InnerVolumeSpecName "kube-api-access-ldjs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:32:27.446863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446829 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-trusted-ca-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:27.446863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446854 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldjs9\" (UniqueName: \"kubernetes.io/projected/66157c67-a610-4adc-9887-442858073a01-kube-api-access-ldjs9\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:27.446863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446863 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-console-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:27.447054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446872 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:27.447054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446881 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-oauth-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:27.447054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446889 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66157c67-a610-4adc-9887-442858073a01-console-oauth-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:27.447054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:27.446897 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66157c67-a610-4adc-9887-442858073a01-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:28.059327 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.059226 2575 generic.go:356] "Generic (PLEG): container finished" podID="66157c67-a610-4adc-9887-442858073a01" containerID="5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b" exitCode=0 Sep 06 19:32:28.059327 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.059289 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d6c95b8f-9qkjn" Sep 06 19:32:28.059327 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.059312 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d6c95b8f-9qkjn" event={"ID":"66157c67-a610-4adc-9887-442858073a01","Type":"ContainerDied","Data":"5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b"} Sep 06 19:32:28.059896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.059384 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d6c95b8f-9qkjn" event={"ID":"66157c67-a610-4adc-9887-442858073a01","Type":"ContainerDied","Data":"898043356e8badf91181193c3ee29de61c1070b78bdc6e1b451ab0911bdeecca"} Sep 06 19:32:28.059896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.059405 2575 scope.go:117] "RemoveContainer" containerID="5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b" Sep 06 19:32:28.067667 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.067649 2575 scope.go:117] "RemoveContainer" containerID="5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b" Sep 06 19:32:28.067975 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:32:28.067954 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b\": container with ID starting with 5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b not found: ID does not exist" containerID="5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b" Sep 06 19:32:28.068036 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.067983 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b"} err="failed to get container status \"5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b\": rpc error: code = NotFound desc = could not find container \"5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b\": container with ID starting with 5b1b98132d3d7fd781c6f2abce5e6405c0a0debdf36277b4412d10234d65319b not found: ID does not exist" Sep 06 19:32:28.074201 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.074180 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77d6c95b8f-9qkjn"] Sep 06 19:32:28.079180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:28.079160 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77d6c95b8f-9qkjn"] Sep 06 19:32:29.761331 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:29.761299 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66157c67-a610-4adc-9887-442858073a01" path="/var/lib/kubelet/pods/66157c67-a610-4adc-9887-442858073a01/volumes" Sep 06 19:32:37.635080 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:37.635054 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:32:37.639275 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:37.639252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:32:37.640902 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:37.640875 2575 kubelet.go:1648] "Image garbage collection succeeded" Sep 06 19:32:39.448324 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.448293 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz"] Sep 06 19:32:39.450582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.448557 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66157c67-a610-4adc-9887-442858073a01" containerName="console" Sep 06 19:32:39.450582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.448568 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66157c67-a610-4adc-9887-442858073a01" containerName="console" Sep 06 19:32:39.450582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.448624 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66157c67-a610-4adc-9887-442858073a01" containerName="console" Sep 06 19:32:39.451434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.451419 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.453856 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.453824 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 19:32:39.453992 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.453830 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 19:32:39.454826 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.454813 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6znk5\"" Sep 06 19:32:39.460087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.460056 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz"] Sep 06 19:32:39.538465 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.538427 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtl26\" (UniqueName: \"kubernetes.io/projected/6a197432-97ae-447f-86b9-90b89e99e1e2-kube-api-access-vtl26\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.538465 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.538464 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.538657 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.538548 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.639270 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.639234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.639403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.639288 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtl26\" (UniqueName: \"kubernetes.io/projected/6a197432-97ae-447f-86b9-90b89e99e1e2-kube-api-access-vtl26\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.639403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.639309 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.639637 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.639619 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.639673 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.639637 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.647513 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.647485 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtl26\" (UniqueName: \"kubernetes.io/projected/6a197432-97ae-447f-86b9-90b89e99e1e2-kube-api-access-vtl26\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:39.761281 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:39.761209 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:40.036556 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:40.036478 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz"] Sep 06 19:32:40.039320 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:32:40.039291 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a197432_97ae_447f_86b9_90b89e99e1e2.slice/crio-c0d692fb0ae0aaa6b3ca34a5552a2ec136aaafa425db3182b7aabcd8a9c30bec WatchSource:0}: Error finding container c0d692fb0ae0aaa6b3ca34a5552a2ec136aaafa425db3182b7aabcd8a9c30bec: Status 404 returned error can't find the container with id c0d692fb0ae0aaa6b3ca34a5552a2ec136aaafa425db3182b7aabcd8a9c30bec Sep 06 19:32:40.041462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:40.041446 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 19:32:40.108054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:40.108020 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" event={"ID":"6a197432-97ae-447f-86b9-90b89e99e1e2","Type":"ContainerStarted","Data":"c0d692fb0ae0aaa6b3ca34a5552a2ec136aaafa425db3182b7aabcd8a9c30bec"} Sep 06 19:32:46.132992 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:46.132952 2575 generic.go:356] "Generic (PLEG): container finished" podID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerID="1e51aa4ec6cc5250c3944ddac4b9db6400f0fe6d5c33cffdf66c616a816f21ef" exitCode=0 Sep 06 19:32:46.133388 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:46.133052 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" event={"ID":"6a197432-97ae-447f-86b9-90b89e99e1e2","Type":"ContainerDied","Data":"1e51aa4ec6cc5250c3944ddac4b9db6400f0fe6d5c33cffdf66c616a816f21ef"} Sep 06 19:32:49.147973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:49.147934 2575 generic.go:356] "Generic (PLEG): container finished" podID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerID="8463082f31d63259e6d7da1afd7e66fb8881f5072e5c7053e52f9c199d5b49be" exitCode=0 Sep 06 19:32:49.148345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:49.148004 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" event={"ID":"6a197432-97ae-447f-86b9-90b89e99e1e2","Type":"ContainerDied","Data":"8463082f31d63259e6d7da1afd7e66fb8881f5072e5c7053e52f9c199d5b49be"} Sep 06 19:32:56.179461 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:56.179429 2575 generic.go:356] "Generic (PLEG): container finished" podID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerID="b93541fab2a1dfb1228a8cd7f3379796998eaf309ff16e3af92c8e046c9210ae" exitCode=0 Sep 06 19:32:56.179860 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:56.179479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" event={"ID":"6a197432-97ae-447f-86b9-90b89e99e1e2","Type":"ContainerDied","Data":"b93541fab2a1dfb1228a8cd7f3379796998eaf309ff16e3af92c8e046c9210ae"} Sep 06 19:32:57.313640 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.313615 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:57.393640 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.393604 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-util\") pod \"6a197432-97ae-447f-86b9-90b89e99e1e2\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " Sep 06 19:32:57.393816 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.393653 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtl26\" (UniqueName: \"kubernetes.io/projected/6a197432-97ae-447f-86b9-90b89e99e1e2-kube-api-access-vtl26\") pod \"6a197432-97ae-447f-86b9-90b89e99e1e2\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " Sep 06 19:32:57.393816 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.393675 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-bundle\") pod \"6a197432-97ae-447f-86b9-90b89e99e1e2\" (UID: \"6a197432-97ae-447f-86b9-90b89e99e1e2\") " Sep 06 19:32:57.394461 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.394432 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-bundle" (OuterVolumeSpecName: "bundle") pod "6a197432-97ae-447f-86b9-90b89e99e1e2" (UID: "6a197432-97ae-447f-86b9-90b89e99e1e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:32:57.395925 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.395901 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a197432-97ae-447f-86b9-90b89e99e1e2-kube-api-access-vtl26" (OuterVolumeSpecName: "kube-api-access-vtl26") pod "6a197432-97ae-447f-86b9-90b89e99e1e2" (UID: "6a197432-97ae-447f-86b9-90b89e99e1e2"). InnerVolumeSpecName "kube-api-access-vtl26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:32:57.398033 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.398008 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-util" (OuterVolumeSpecName: "util") pod "6a197432-97ae-447f-86b9-90b89e99e1e2" (UID: "6a197432-97ae-447f-86b9-90b89e99e1e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:32:57.494950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.494866 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:57.494950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.494894 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtl26\" (UniqueName: \"kubernetes.io/projected/6a197432-97ae-447f-86b9-90b89e99e1e2-kube-api-access-vtl26\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:57.494950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:57.494905 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a197432-97ae-447f-86b9-90b89e99e1e2-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:32:58.189246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:58.189210 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" event={"ID":"6a197432-97ae-447f-86b9-90b89e99e1e2","Type":"ContainerDied","Data":"c0d692fb0ae0aaa6b3ca34a5552a2ec136aaafa425db3182b7aabcd8a9c30bec"} Sep 06 19:32:58.189246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:58.189234 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z6mtz" Sep 06 19:32:58.189518 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:32:58.189266 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0d692fb0ae0aaa6b3ca34a5552a2ec136aaafa425db3182b7aabcd8a9c30bec" Sep 06 19:33:02.339840 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.339808 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt"] Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340095 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="util" Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340106 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="util" Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340122 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="pull" Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340127 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="pull" Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340134 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="extract" Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340139 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="extract" Sep 06 19:33:02.340240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.340189 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a197432-97ae-447f-86b9-90b89e99e1e2" containerName="extract" Sep 06 19:33:02.379854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.379810 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt"] Sep 06 19:33:02.380018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.379910 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.383926 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.383906 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 06 19:33:02.384247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.384231 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 06 19:33:02.384556 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.384537 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cz8cn\"" Sep 06 19:33:02.432724 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.432688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krthl\" (UniqueName: \"kubernetes.io/projected/32d4fcf7-ae8e-45a5-9b41-9400fc5c874b-kube-api-access-krthl\") pod \"cert-manager-operator-controller-manager-55c444bf86-nvzdt\" (UID: \"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.432724 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.432728 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32d4fcf7-ae8e-45a5-9b41-9400fc5c874b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nvzdt\" (UID: \"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.533740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.533706 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krthl\" (UniqueName: \"kubernetes.io/projected/32d4fcf7-ae8e-45a5-9b41-9400fc5c874b-kube-api-access-krthl\") pod \"cert-manager-operator-controller-manager-55c444bf86-nvzdt\" (UID: \"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.533740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.533764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32d4fcf7-ae8e-45a5-9b41-9400fc5c874b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nvzdt\" (UID: \"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.534100 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.534085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32d4fcf7-ae8e-45a5-9b41-9400fc5c874b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nvzdt\" (UID: \"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.541848 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.541813 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krthl\" (UniqueName: \"kubernetes.io/projected/32d4fcf7-ae8e-45a5-9b41-9400fc5c874b-kube-api-access-krthl\") pod \"cert-manager-operator-controller-manager-55c444bf86-nvzdt\" (UID: \"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.689178 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.689150 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" Sep 06 19:33:02.988269 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:02.988245 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt"] Sep 06 19:33:02.991020 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:02.990990 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d4fcf7_ae8e_45a5_9b41_9400fc5c874b.slice/crio-b103f4102b3c5b04e54f45ecc688df3859983158aa34eb891b98b477c8815dd0 WatchSource:0}: Error finding container b103f4102b3c5b04e54f45ecc688df3859983158aa34eb891b98b477c8815dd0: Status 404 returned error can't find the container with id b103f4102b3c5b04e54f45ecc688df3859983158aa34eb891b98b477c8815dd0 Sep 06 19:33:03.208210 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:03.208172 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" event={"ID":"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b","Type":"ContainerStarted","Data":"b103f4102b3c5b04e54f45ecc688df3859983158aa34eb891b98b477c8815dd0"} Sep 06 19:33:05.218877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:05.218843 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" event={"ID":"32d4fcf7-ae8e-45a5-9b41-9400fc5c874b","Type":"ContainerStarted","Data":"fbcf21c995d49745e2c7da191cfaf00c322e4c87426a9e97ff83cbc90791dcfe"} Sep 06 19:33:05.241403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:05.241353 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nvzdt" podStartSLOduration=1.661816093 podStartE2EDuration="3.241336809s" totalImagesPullingTime="1.579520716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:05.239661248 +0000 UTC m=+327.974265982" watchObservedRunningTime="2026-09-06 19:33:05.241336809 +0000 UTC m=+327.975941542" Sep 06 19:33:06.425716 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.425675 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2"] Sep 06 19:33:06.428460 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.428437 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.430715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.430695 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 19:33:06.430835 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.430772 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 19:33:06.431647 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.431632 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6znk5\"" Sep 06 19:33:06.445460 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.445442 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2"] Sep 06 19:33:06.464646 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.464618 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptc7h\" (UniqueName: \"kubernetes.io/projected/83451e4e-0922-4f45-9ac0-58f2caa508d4-kube-api-access-ptc7h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.464772 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.464653 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.464772 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.464726 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.565848 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.565814 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptc7h\" (UniqueName: \"kubernetes.io/projected/83451e4e-0922-4f45-9ac0-58f2caa508d4-kube-api-access-ptc7h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.565992 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.565852 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.565992 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.565889 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.566203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.566180 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.566284 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.566261 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.574352 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.574322 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptc7h\" (UniqueName: \"kubernetes.io/projected/83451e4e-0922-4f45-9ac0-58f2caa508d4-kube-api-access-ptc7h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:06.737946 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:06.737861 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:07.005995 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:07.005920 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2"] Sep 06 19:33:07.008793 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:07.008742 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83451e4e_0922_4f45_9ac0_58f2caa508d4.slice/crio-8e54b65e5c5da41396d4dcda2807f5951abd177a069740cee4f2f03e74adb150 WatchSource:0}: Error finding container 8e54b65e5c5da41396d4dcda2807f5951abd177a069740cee4f2f03e74adb150: Status 404 returned error can't find the container with id 8e54b65e5c5da41396d4dcda2807f5951abd177a069740cee4f2f03e74adb150 Sep 06 19:33:07.228084 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:07.228052 2575 generic.go:356] "Generic (PLEG): container finished" podID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerID="253227813db31b47d4e9d61933e00f4d565a87e5d13ac4ea64fc7e0e062cf674" exitCode=0 Sep 06 19:33:07.228271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:07.228147 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" event={"ID":"83451e4e-0922-4f45-9ac0-58f2caa508d4","Type":"ContainerDied","Data":"253227813db31b47d4e9d61933e00f4d565a87e5d13ac4ea64fc7e0e062cf674"} Sep 06 19:33:07.228271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:07.228196 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" event={"ID":"83451e4e-0922-4f45-9ac0-58f2caa508d4","Type":"ContainerStarted","Data":"8e54b65e5c5da41396d4dcda2807f5951abd177a069740cee4f2f03e74adb150"} Sep 06 19:33:10.244727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:10.244690 2575 generic.go:356] "Generic (PLEG): container finished" podID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerID="f8e1359bfe96bd6958f8d8f2cd3595967d133b980497c4bbb3868d051085551a" exitCode=0 Sep 06 19:33:10.245085 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:10.244781 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" event={"ID":"83451e4e-0922-4f45-9ac0-58f2caa508d4","Type":"ContainerDied","Data":"f8e1359bfe96bd6958f8d8f2cd3595967d133b980497c4bbb3868d051085551a"} Sep 06 19:33:11.250912 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:11.250872 2575 generic.go:356] "Generic (PLEG): container finished" podID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerID="217dcc5f5c54e907d5d35a530d8eebaec7433b712f1b1d9146375b7d04f83e46" exitCode=0 Sep 06 19:33:11.251272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:11.250959 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" event={"ID":"83451e4e-0922-4f45-9ac0-58f2caa508d4","Type":"ContainerDied","Data":"217dcc5f5c54e907d5d35a530d8eebaec7433b712f1b1d9146375b7d04f83e46"} Sep 06 19:33:12.377272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.377246 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:12.517820 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.517709 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-util\") pod \"83451e4e-0922-4f45-9ac0-58f2caa508d4\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " Sep 06 19:33:12.517820 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.517805 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptc7h\" (UniqueName: \"kubernetes.io/projected/83451e4e-0922-4f45-9ac0-58f2caa508d4-kube-api-access-ptc7h\") pod \"83451e4e-0922-4f45-9ac0-58f2caa508d4\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " Sep 06 19:33:12.518001 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.517872 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-bundle\") pod \"83451e4e-0922-4f45-9ac0-58f2caa508d4\" (UID: \"83451e4e-0922-4f45-9ac0-58f2caa508d4\") " Sep 06 19:33:12.518262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.518229 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-bundle" (OuterVolumeSpecName: "bundle") pod "83451e4e-0922-4f45-9ac0-58f2caa508d4" (UID: "83451e4e-0922-4f45-9ac0-58f2caa508d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:33:12.520004 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.519981 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83451e4e-0922-4f45-9ac0-58f2caa508d4-kube-api-access-ptc7h" (OuterVolumeSpecName: "kube-api-access-ptc7h") pod "83451e4e-0922-4f45-9ac0-58f2caa508d4" (UID: "83451e4e-0922-4f45-9ac0-58f2caa508d4"). InnerVolumeSpecName "kube-api-access-ptc7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:33:12.522961 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.522924 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-util" (OuterVolumeSpecName: "util") pod "83451e4e-0922-4f45-9ac0-58f2caa508d4" (UID: "83451e4e-0922-4f45-9ac0-58f2caa508d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:33:12.619002 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.618968 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:12.619002 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.618994 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptc7h\" (UniqueName: \"kubernetes.io/projected/83451e4e-0922-4f45-9ac0-58f2caa508d4-kube-api-access-ptc7h\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:12.619002 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:12.619004 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83451e4e-0922-4f45-9ac0-58f2caa508d4-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:13.262545 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.262511 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" event={"ID":"83451e4e-0922-4f45-9ac0-58f2caa508d4","Type":"ContainerDied","Data":"8e54b65e5c5da41396d4dcda2807f5951abd177a069740cee4f2f03e74adb150"} Sep 06 19:33:13.262713 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.262564 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e54b65e5c5da41396d4dcda2807f5951abd177a069740cee4f2f03e74adb150" Sep 06 19:33:13.262713 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.262525 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8grc2" Sep 06 19:33:13.648865 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.648779 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx"] Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649065 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="extract" Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649078 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="extract" Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649088 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="util" Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649093 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="util" Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649111 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="pull" Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649117 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="pull" Sep 06 19:33:13.649253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.649167 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="83451e4e-0922-4f45-9ac0-58f2caa508d4" containerName="extract" Sep 06 19:33:13.650979 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.650964 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.654051 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.654030 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-st4ks\"" Sep 06 19:33:13.655032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.655012 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 06 19:33:13.655114 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.655049 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 06 19:33:13.660097 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.660076 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx"] Sep 06 19:33:13.728700 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.728662 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8ch9\" (UniqueName: \"kubernetes.io/projected/09513559-1e4b-4aa8-839a-5736fcaee51d-kube-api-access-k8ch9\") pod \"cert-manager-cainjector-559cdd8c87-4pvgx\" (UID: \"09513559-1e4b-4aa8-839a-5736fcaee51d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.728895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.728708 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09513559-1e4b-4aa8-839a-5736fcaee51d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4pvgx\" (UID: \"09513559-1e4b-4aa8-839a-5736fcaee51d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.829674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.829644 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8ch9\" (UniqueName: \"kubernetes.io/projected/09513559-1e4b-4aa8-839a-5736fcaee51d-kube-api-access-k8ch9\") pod \"cert-manager-cainjector-559cdd8c87-4pvgx\" (UID: \"09513559-1e4b-4aa8-839a-5736fcaee51d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.829902 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.829697 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09513559-1e4b-4aa8-839a-5736fcaee51d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4pvgx\" (UID: \"09513559-1e4b-4aa8-839a-5736fcaee51d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.837197 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.837163 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8ch9\" (UniqueName: \"kubernetes.io/projected/09513559-1e4b-4aa8-839a-5736fcaee51d-kube-api-access-k8ch9\") pod \"cert-manager-cainjector-559cdd8c87-4pvgx\" (UID: \"09513559-1e4b-4aa8-839a-5736fcaee51d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.837311 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.837165 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09513559-1e4b-4aa8-839a-5736fcaee51d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4pvgx\" (UID: \"09513559-1e4b-4aa8-839a-5736fcaee51d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:13.966238 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:13.966195 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" Sep 06 19:33:14.238040 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:14.237965 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx"] Sep 06 19:33:14.240986 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:14.240957 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09513559_1e4b_4aa8_839a_5736fcaee51d.slice/crio-2d361a58394ec66997bfb4d667c27435680670177849e83bee64d2a0cb6ca9b1 WatchSource:0}: Error finding container 2d361a58394ec66997bfb4d667c27435680670177849e83bee64d2a0cb6ca9b1: Status 404 returned error can't find the container with id 2d361a58394ec66997bfb4d667c27435680670177849e83bee64d2a0cb6ca9b1 Sep 06 19:33:14.268120 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:14.268085 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" event={"ID":"09513559-1e4b-4aa8-839a-5736fcaee51d","Type":"ContainerStarted","Data":"2d361a58394ec66997bfb4d667c27435680670177849e83bee64d2a0cb6ca9b1"} Sep 06 19:33:17.284676 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:17.284633 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" event={"ID":"09513559-1e4b-4aa8-839a-5736fcaee51d","Type":"ContainerStarted","Data":"4a27a922d854350ea356ec83d57664e590e5241c7c5d19c1681e607afd4d3f56"} Sep 06 19:33:17.299839 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:17.299791 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4pvgx" podStartSLOduration=1.9544601400000001 podStartE2EDuration="4.29977449s" totalImagesPullingTime="2.34531435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:17.298268421 +0000 UTC m=+340.032873154" watchObservedRunningTime="2026-09-06 19:33:17.29977449 +0000 UTC m=+340.034379218" Sep 06 19:33:26.812641 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.812609 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265"] Sep 06 19:33:26.815034 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.815015 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:26.817384 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.817362 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 19:33:26.818174 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.818158 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 19:33:26.818266 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.818163 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6znk5\"" Sep 06 19:33:26.822816 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.822795 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265"] Sep 06 19:33:26.944115 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.944072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:26.944115 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.944121 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qh82\" (UniqueName: \"kubernetes.io/projected/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-kube-api-access-7qh82\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:26.944331 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:26.944200 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.045071 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.045036 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.045071 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.045072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qh82\" (UniqueName: \"kubernetes.io/projected/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-kube-api-access-7qh82\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.045283 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.045125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.045471 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.045451 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.045507 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.045488 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.052612 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.052580 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qh82\" (UniqueName: \"kubernetes.io/projected/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-kube-api-access-7qh82\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.124777 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.124699 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:27.392215 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:27.392192 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265"] Sep 06 19:33:27.394675 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:27.394648 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1451ff49_8a39_4b2e_a8db_63bd5d8d4b6f.slice/crio-0bab69b02ca1dcecaa9c1762237c8827cf5c8d4c20176976b66d2eba65d756de WatchSource:0}: Error finding container 0bab69b02ca1dcecaa9c1762237c8827cf5c8d4c20176976b66d2eba65d756de: Status 404 returned error can't find the container with id 0bab69b02ca1dcecaa9c1762237c8827cf5c8d4c20176976b66d2eba65d756de Sep 06 19:33:28.337856 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:28.337727 2575 generic.go:356] "Generic (PLEG): container finished" podID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerID="31ed03992386e3404547d19d56a59a88adb6ab031e474a9dee87ff2b1d5f93c5" exitCode=0 Sep 06 19:33:28.337856 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:28.337821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" event={"ID":"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f","Type":"ContainerDied","Data":"31ed03992386e3404547d19d56a59a88adb6ab031e474a9dee87ff2b1d5f93c5"} Sep 06 19:33:28.338292 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:28.337877 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" event={"ID":"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f","Type":"ContainerStarted","Data":"0bab69b02ca1dcecaa9c1762237c8827cf5c8d4c20176976b66d2eba65d756de"} Sep 06 19:33:29.343591 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:29.343553 2575 generic.go:356] "Generic (PLEG): container finished" podID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerID="96d3ccea64892ddc0769c98d8a79e2aa91c3cc78ca8182a6689c836fcbfbe927" exitCode=0 Sep 06 19:33:29.343998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:29.343625 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" event={"ID":"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f","Type":"ContainerDied","Data":"96d3ccea64892ddc0769c98d8a79e2aa91c3cc78ca8182a6689c836fcbfbe927"} Sep 06 19:33:30.350424 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:30.350388 2575 generic.go:356] "Generic (PLEG): container finished" podID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerID="27732da773f9d7158cf49f03b1a6f301632e669130f2aebaeaae4917db5f9872" exitCode=0 Sep 06 19:33:30.350882 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:30.350454 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" event={"ID":"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f","Type":"ContainerDied","Data":"27732da773f9d7158cf49f03b1a6f301632e669130f2aebaeaae4917db5f9872"} Sep 06 19:33:31.499819 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.499786 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:31.582008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.581982 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qh82\" (UniqueName: \"kubernetes.io/projected/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-kube-api-access-7qh82\") pod \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " Sep 06 19:33:31.582008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.582014 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-util\") pod \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " Sep 06 19:33:31.582249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.582037 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-bundle\") pod \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\" (UID: \"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f\") " Sep 06 19:33:31.582725 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.582697 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-bundle" (OuterVolumeSpecName: "bundle") pod "1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" (UID: "1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:33:31.584238 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.584216 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-kube-api-access-7qh82" (OuterVolumeSpecName: "kube-api-access-7qh82") pod "1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" (UID: "1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f"). InnerVolumeSpecName "kube-api-access-7qh82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:33:31.590243 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.590218 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-util" (OuterVolumeSpecName: "util") pod "1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" (UID: "1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:33:31.683462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.683433 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qh82\" (UniqueName: \"kubernetes.io/projected/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-kube-api-access-7qh82\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:31.683462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.683463 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:31.683659 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:31.683475 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:32.360957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:32.360924 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" event={"ID":"1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f","Type":"ContainerDied","Data":"0bab69b02ca1dcecaa9c1762237c8827cf5c8d4c20176976b66d2eba65d756de"} Sep 06 19:33:32.361129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:32.360976 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjh265" Sep 06 19:33:32.361129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:32.360977 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bab69b02ca1dcecaa9c1762237c8827cf5c8d4c20176976b66d2eba65d756de" Sep 06 19:33:36.232838 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.232800 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2"] Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233075 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="util" Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233086 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="util" Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233099 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="extract" Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233105 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="extract" Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233114 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="pull" Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233120 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="pull" Sep 06 19:33:36.233213 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.233183 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1451ff49-8a39-4b2e-a8db-63bd5d8d4b6f" containerName="extract" Sep 06 19:33:36.235332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.235312 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.237626 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.237603 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 19:33:36.238487 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.238469 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 19:33:36.238538 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.238494 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6znk5\"" Sep 06 19:33:36.244214 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.244191 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2"] Sep 06 19:33:36.321817 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.321787 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.321991 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.321832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ww5\" (UniqueName: \"kubernetes.io/projected/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-kube-api-access-n2ww5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.321991 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.321883 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.423060 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.423029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.423259 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.423067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ww5\" (UniqueName: \"kubernetes.io/projected/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-kube-api-access-n2ww5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.423259 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.423097 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.423498 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.423478 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.423566 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.423484 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.430537 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.430510 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ww5\" (UniqueName: \"kubernetes.io/projected/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-kube-api-access-n2ww5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.544479 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.544397 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:36.821167 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:36.821137 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2"] Sep 06 19:33:36.823662 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:36.823633 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6f9a02_52a6_4db9_a3d9_1f21b451870b.slice/crio-c07de587667cb6e4130d9a3de8342cf7c148794ee4a7977b10d8e7c6b8465ac5 WatchSource:0}: Error finding container c07de587667cb6e4130d9a3de8342cf7c148794ee4a7977b10d8e7c6b8465ac5: Status 404 returned error can't find the container with id c07de587667cb6e4130d9a3de8342cf7c148794ee4a7977b10d8e7c6b8465ac5 Sep 06 19:33:37.384707 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:37.384676 2575 generic.go:356] "Generic (PLEG): container finished" podID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerID="e06c3706ee66ba4167e2a7dfb07b6fc73c41e182d9df87abb06fb660be603559" exitCode=0 Sep 06 19:33:37.385129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:37.384724 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" event={"ID":"aa6f9a02-52a6-4db9-a3d9-1f21b451870b","Type":"ContainerDied","Data":"e06c3706ee66ba4167e2a7dfb07b6fc73c41e182d9df87abb06fb660be603559"} Sep 06 19:33:37.385129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:37.384777 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" event={"ID":"aa6f9a02-52a6-4db9-a3d9-1f21b451870b","Type":"ContainerStarted","Data":"c07de587667cb6e4130d9a3de8342cf7c148794ee4a7977b10d8e7c6b8465ac5"} Sep 06 19:33:38.391828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.391712 2575 generic.go:356] "Generic (PLEG): container finished" podID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerID="61bc2d508e15a18ede0202e538369670c2be4c2de1a5c69b5af1619873652f07" exitCode=0 Sep 06 19:33:38.391828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.391794 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" event={"ID":"aa6f9a02-52a6-4db9-a3d9-1f21b451870b","Type":"ContainerDied","Data":"61bc2d508e15a18ede0202e538369670c2be4c2de1a5c69b5af1619873652f07"} Sep 06 19:33:38.526403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.526364 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr"] Sep 06 19:33:38.529887 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.529860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.532908 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.532820 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 06 19:33:38.532908 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.532871 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 06 19:33:38.533113 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.532871 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 06 19:33:38.533113 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.532876 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pp5zq\"" Sep 06 19:33:38.533113 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.532952 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 06 19:33:38.545739 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.545718 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr"] Sep 06 19:33:38.643244 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.643162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4be8c382-9dd0-4314-b57e-fe0997aa05eb-tmp\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.643244 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.643212 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4be8c382-9dd0-4314-b57e-fe0997aa05eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.643420 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.643261 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqk7n\" (UniqueName: \"kubernetes.io/projected/4be8c382-9dd0-4314-b57e-fe0997aa05eb-kube-api-access-cqk7n\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.643420 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.643326 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4be8c382-9dd0-4314-b57e-fe0997aa05eb-manifests\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.643420 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.643350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4be8c382-9dd0-4314-b57e-fe0997aa05eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.744689 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.744656 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4be8c382-9dd0-4314-b57e-fe0997aa05eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.744689 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.744693 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqk7n\" (UniqueName: \"kubernetes.io/projected/4be8c382-9dd0-4314-b57e-fe0997aa05eb-kube-api-access-cqk7n\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.744928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.744741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4be8c382-9dd0-4314-b57e-fe0997aa05eb-manifests\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.744928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.744783 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4be8c382-9dd0-4314-b57e-fe0997aa05eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.744928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.744833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4be8c382-9dd0-4314-b57e-fe0997aa05eb-tmp\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.745253 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.745222 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4be8c382-9dd0-4314-b57e-fe0997aa05eb-manifests\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.747336 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.747312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4be8c382-9dd0-4314-b57e-fe0997aa05eb-tmp\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.747436 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.747364 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4be8c382-9dd0-4314-b57e-fe0997aa05eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.747436 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.747405 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4be8c382-9dd0-4314-b57e-fe0997aa05eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.756346 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.756312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqk7n\" (UniqueName: \"kubernetes.io/projected/4be8c382-9dd0-4314-b57e-fe0997aa05eb-kube-api-access-cqk7n\") pod \"opendatahub-operator-controller-manager-7f964dcc99-2fbcr\" (UID: \"4be8c382-9dd0-4314-b57e-fe0997aa05eb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:38.892916 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:38.892876 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:39.139346 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.139314 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769675c48-b565m"] Sep 06 19:33:39.142875 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.142854 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.146101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.146039 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 06 19:33:39.146101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.146088 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 06 19:33:39.146252 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.146051 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 06 19:33:39.146252 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.146124 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 06 19:33:39.146252 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.146186 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 06 19:33:39.146766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.146730 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9mh7l\"" Sep 06 19:33:39.156394 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.156369 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769675c48-b565m"] Sep 06 19:33:39.184664 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.184637 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr"] Sep 06 19:33:39.187134 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:39.187107 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be8c382_9dd0_4314_b57e_fe0997aa05eb.slice/crio-f1daf36f3a3b2065be416ca6a6d9123ba8f4b3f79954fd7920b1cd8a87d2b838 WatchSource:0}: Error finding container f1daf36f3a3b2065be416ca6a6d9123ba8f4b3f79954fd7920b1cd8a87d2b838: Status 404 returned error can't find the container with id f1daf36f3a3b2065be416ca6a6d9123ba8f4b3f79954fd7920b1cd8a87d2b838 Sep 06 19:33:39.249878 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.249843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-manager-config\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.250052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.249903 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-metrics-cert\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.250052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.249960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6vc7\" (UniqueName: \"kubernetes.io/projected/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-kube-api-access-v6vc7\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.250052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.249978 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-cert\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.350933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.350901 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6vc7\" (UniqueName: \"kubernetes.io/projected/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-kube-api-access-v6vc7\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.350933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.350936 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-cert\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.351163 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.350956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-manager-config\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.351163 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.351002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-metrics-cert\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.351704 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.351678 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-manager-config\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.353535 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.353505 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-cert\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.353535 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.353529 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-metrics-cert\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.358922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.358904 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6vc7\" (UniqueName: \"kubernetes.io/projected/0e34c8f9-5d4d-41cd-9bce-435e6cbf937e-kube-api-access-v6vc7\") pod \"lws-controller-manager-7769675c48-b565m\" (UID: \"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e\") " pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.398901 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.398828 2575 generic.go:356] "Generic (PLEG): container finished" podID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerID="a6c5b3da9c52b0f611c62fd6347aa2040d85c435a99870d440677badbb697a87" exitCode=0 Sep 06 19:33:39.399228 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.398916 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" event={"ID":"aa6f9a02-52a6-4db9-a3d9-1f21b451870b","Type":"ContainerDied","Data":"a6c5b3da9c52b0f611c62fd6347aa2040d85c435a99870d440677badbb697a87"} Sep 06 19:33:39.400243 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.400224 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" event={"ID":"4be8c382-9dd0-4314-b57e-fe0997aa05eb","Type":"ContainerStarted","Data":"f1daf36f3a3b2065be416ca6a6d9123ba8f4b3f79954fd7920b1cd8a87d2b838"} Sep 06 19:33:39.452890 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.452850 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:39.730531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:39.730500 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769675c48-b565m"] Sep 06 19:33:39.735180 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:33:39.735148 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e34c8f9_5d4d_41cd_9bce_435e6cbf937e.slice/crio-d620c761dbddf9091762fe27cfd7ba70ad3226fe3dc8e91a0eee4e0a8881af22 WatchSource:0}: Error finding container d620c761dbddf9091762fe27cfd7ba70ad3226fe3dc8e91a0eee4e0a8881af22: Status 404 returned error can't find the container with id d620c761dbddf9091762fe27cfd7ba70ad3226fe3dc8e91a0eee4e0a8881af22 Sep 06 19:33:40.407830 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:40.407794 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" event={"ID":"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e","Type":"ContainerStarted","Data":"d620c761dbddf9091762fe27cfd7ba70ad3226fe3dc8e91a0eee4e0a8881af22"} Sep 06 19:33:41.163154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.162853 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:41.269360 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.269327 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2ww5\" (UniqueName: \"kubernetes.io/projected/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-kube-api-access-n2ww5\") pod \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " Sep 06 19:33:41.269545 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.269376 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-util\") pod \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " Sep 06 19:33:41.269545 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.269498 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-bundle\") pod \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\" (UID: \"aa6f9a02-52a6-4db9-a3d9-1f21b451870b\") " Sep 06 19:33:41.270454 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.270401 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-bundle" (OuterVolumeSpecName: "bundle") pod "aa6f9a02-52a6-4db9-a3d9-1f21b451870b" (UID: "aa6f9a02-52a6-4db9-a3d9-1f21b451870b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:33:41.272050 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.272009 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-kube-api-access-n2ww5" (OuterVolumeSpecName: "kube-api-access-n2ww5") pod "aa6f9a02-52a6-4db9-a3d9-1f21b451870b" (UID: "aa6f9a02-52a6-4db9-a3d9-1f21b451870b"). InnerVolumeSpecName "kube-api-access-n2ww5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:33:41.275881 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.275853 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-util" (OuterVolumeSpecName: "util") pod "aa6f9a02-52a6-4db9-a3d9-1f21b451870b" (UID: "aa6f9a02-52a6-4db9-a3d9-1f21b451870b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:33:41.371217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.371176 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2ww5\" (UniqueName: \"kubernetes.io/projected/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-kube-api-access-n2ww5\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:41.371217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.371216 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:41.371465 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.371232 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa6f9a02-52a6-4db9-a3d9-1f21b451870b-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:33:41.417428 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.417389 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" event={"ID":"aa6f9a02-52a6-4db9-a3d9-1f21b451870b","Type":"ContainerDied","Data":"c07de587667cb6e4130d9a3de8342cf7c148794ee4a7977b10d8e7c6b8465ac5"} Sep 06 19:33:41.417870 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.417491 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c07de587667cb6e4130d9a3de8342cf7c148794ee4a7977b10d8e7c6b8465ac5" Sep 06 19:33:41.417870 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:41.417569 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d4hk2" Sep 06 19:33:43.428607 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:43.428570 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" event={"ID":"4be8c382-9dd0-4314-b57e-fe0997aa05eb","Type":"ContainerStarted","Data":"91bc333aea4758b83ba1cbc0cf3d03314cadac5a0612baa1704f6c0ef11e1f10"} Sep 06 19:33:43.430162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:43.430139 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" event={"ID":"0e34c8f9-5d4d-41cd-9bce-435e6cbf937e","Type":"ContainerStarted","Data":"51d8b3c2ad1f982160b0be3562429773981823a707a55c10691c8254fc7eecfb"} Sep 06 19:33:43.430281 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:43.430190 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:43.462129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:43.462078 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" podStartSLOduration=1.621363792 podStartE2EDuration="4.462061306s" totalImagesPullingTime="2.840697514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:43.461553791 +0000 UTC m=+366.196158523" watchObservedRunningTime="2026-09-06 19:33:43.462061306 +0000 UTC m=+366.196666039" Sep 06 19:33:44.436251 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:44.436216 2575 generic.go:356] "Generic (PLEG): container finished" podID="4be8c382-9dd0-4314-b57e-fe0997aa05eb" containerID="91bc333aea4758b83ba1cbc0cf3d03314cadac5a0612baa1704f6c0ef11e1f10" exitCode=0 Sep 06 19:33:44.436704 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:44.436309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" event={"ID":"4be8c382-9dd0-4314-b57e-fe0997aa05eb","Type":"ContainerDied","Data":"91bc333aea4758b83ba1cbc0cf3d03314cadac5a0612baa1704f6c0ef11e1f10"} Sep 06 19:33:45.442660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:45.442622 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" event={"ID":"4be8c382-9dd0-4314-b57e-fe0997aa05eb","Type":"ContainerStarted","Data":"47dcee8eda472576d93350901014c71d906cfc3b325ce9dbe6ab15c4421a5dd4"} Sep 06 19:33:45.443081 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:45.442791 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:33:45.462410 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:45.462362 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" podStartSLOduration=2.7560883499999997 podStartE2EDuration="7.462344878s" totalImagesPullingTime="3.657687142s" totalInitContainerRuntime="1.048569386s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:33:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:33:45.460563842 +0000 UTC m=+368.195168598" watchObservedRunningTime="2026-09-06 19:33:45.462344878 +0000 UTC m=+368.196949668" Sep 06 19:33:54.439019 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:54.438987 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7769675c48-b565m" Sep 06 19:33:56.450294 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:33:56.450266 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f964dcc99-2fbcr" Sep 06 19:34:05.452304 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452267 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bc997b6c8-mz6b6"] Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452553 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="util" Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452564 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="util" Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452579 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="pull" Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452586 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="pull" Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452596 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="extract" Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452601 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="extract" Sep 06 19:34:05.452783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.452654 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6f9a02-52a6-4db9-a3d9-1f21b451870b" containerName="extract" Sep 06 19:34:05.455987 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.455967 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.478915 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.478886 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc997b6c8-mz6b6"] Sep 06 19:34:05.557430 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557400 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-trusted-ca-bundle\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.557585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-serving-cert\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.557585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-config\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.557585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557485 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj8tm\" (UniqueName: \"kubernetes.io/projected/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-kube-api-access-cj8tm\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.557585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557500 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-oauth-serving-cert\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.557585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-service-ca\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.557585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.557536 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-oauth-config\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658695 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-trusted-ca-bundle\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658708 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-serving-cert\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-config\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658809 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj8tm\" (UniqueName: \"kubernetes.io/projected/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-kube-api-access-cj8tm\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-oauth-serving-cert\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658859 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-service-ca\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.658903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.658883 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-oauth-config\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.659717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.659649 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-service-ca\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.660083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.660052 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-config\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.660200 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.660139 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-oauth-serving-cert\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.660591 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.660566 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-trusted-ca-bundle\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.661763 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.661666 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-serving-cert\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.661763 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.661722 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-console-oauth-config\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.668083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.668059 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj8tm\" (UniqueName: \"kubernetes.io/projected/0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3-kube-api-access-cj8tm\") pod \"console-6bc997b6c8-mz6b6\" (UID: \"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3\") " pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:05.767337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:05.767248 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:06.043693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:06.043669 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc997b6c8-mz6b6"] Sep 06 19:34:06.045906 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:34:06.045879 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5d9b7c_299d_4dcb_b02e_de9e90ab0ce3.slice/crio-4d6061f6d2d08889721d2df73f2561447981876061678942e931e28b2b363900 WatchSource:0}: Error finding container 4d6061f6d2d08889721d2df73f2561447981876061678942e931e28b2b363900: Status 404 returned error can't find the container with id 4d6061f6d2d08889721d2df73f2561447981876061678942e931e28b2b363900 Sep 06 19:34:06.536509 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:06.536474 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc997b6c8-mz6b6" event={"ID":"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3","Type":"ContainerStarted","Data":"044cf161e2ca523ab1d6c38e1ba25afd507a0ef7d1d713c7fadecbe572d43278"} Sep 06 19:34:06.536921 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:06.536528 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc997b6c8-mz6b6" event={"ID":"0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3","Type":"ContainerStarted","Data":"4d6061f6d2d08889721d2df73f2561447981876061678942e931e28b2b363900"} Sep 06 19:34:06.553840 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:06.553793 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bc997b6c8-mz6b6" podStartSLOduration=1.55377839 podStartE2EDuration="1.55377839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:34:06.552121013 +0000 UTC m=+389.286725776" watchObservedRunningTime="2026-09-06 19:34:06.55377839 +0000 UTC m=+389.288383122" Sep 06 19:34:09.239477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.239440 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hrrwn"] Sep 06 19:34:09.242791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.242773 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.245059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.245036 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 06 19:34:09.245263 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.245245 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-drfsm\"" Sep 06 19:34:09.253332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.253310 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hrrwn"] Sep 06 19:34:09.290500 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.290473 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cee458-63cf-48ee-a6b4-ee109ef946ed-cert\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.290500 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.290503 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtcl\" (UniqueName: \"kubernetes.io/projected/e6cee458-63cf-48ee-a6b4-ee109ef946ed-kube-api-access-fgtcl\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.392008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.391979 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cee458-63cf-48ee-a6b4-ee109ef946ed-cert\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.392172 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.392013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtcl\" (UniqueName: \"kubernetes.io/projected/e6cee458-63cf-48ee-a6b4-ee109ef946ed-kube-api-access-fgtcl\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.392172 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:34:09.392140 2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 06 19:34:09.392244 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:34:09.392214 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6cee458-63cf-48ee-a6b4-ee109ef946ed-cert podName:e6cee458-63cf-48ee-a6b4-ee109ef946ed nodeName:}" failed. No retries permitted until 2026-09-06 19:34:09.892193323 +0000 UTC m=+392.626798049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6cee458-63cf-48ee-a6b4-ee109ef946ed-cert") pod "odh-model-controller-7998c67dc4-hrrwn" (UID: "e6cee458-63cf-48ee-a6b4-ee109ef946ed") : secret "odh-model-controller-webhook-cert" not found Sep 06 19:34:09.402181 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.402158 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtcl\" (UniqueName: \"kubernetes.io/projected/e6cee458-63cf-48ee-a6b4-ee109ef946ed-kube-api-access-fgtcl\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.746270 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.746236 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4gtxj"] Sep 06 19:34:09.750613 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.750593 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:09.752903 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.752887 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4mbmd\"" Sep 06 19:34:09.752969 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.752930 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 06 19:34:09.767267 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.766235 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4gtxj"] Sep 06 19:34:09.794982 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.794952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/742cb1c3-7009-4642-bc2c-f99378b5d102-tls-certs\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:09.795106 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.795011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djjdg\" (UniqueName: \"kubernetes.io/projected/742cb1c3-7009-4642-bc2c-f99378b5d102-kube-api-access-djjdg\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:09.895959 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.895902 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djjdg\" (UniqueName: \"kubernetes.io/projected/742cb1c3-7009-4642-bc2c-f99378b5d102-kube-api-access-djjdg\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:09.896167 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.896004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cee458-63cf-48ee-a6b4-ee109ef946ed-cert\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.896167 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.896043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/742cb1c3-7009-4642-bc2c-f99378b5d102-tls-certs\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:09.896167 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:34:09.896147 2575 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 06 19:34:09.896338 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:34:09.896226 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/742cb1c3-7009-4642-bc2c-f99378b5d102-tls-certs podName:742cb1c3-7009-4642-bc2c-f99378b5d102 nodeName:}" failed. No retries permitted until 2026-09-06 19:34:10.396206072 +0000 UTC m=+393.130810783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/742cb1c3-7009-4642-bc2c-f99378b5d102-tls-certs") pod "model-serving-api-54d89bd5fc-4gtxj" (UID: "742cb1c3-7009-4642-bc2c-f99378b5d102") : secret "model-serving-api-tls" not found Sep 06 19:34:09.898594 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.898570 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cee458-63cf-48ee-a6b4-ee109ef946ed-cert\") pod \"odh-model-controller-7998c67dc4-hrrwn\" (UID: \"e6cee458-63cf-48ee-a6b4-ee109ef946ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:09.903832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:09.903805 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djjdg\" (UniqueName: \"kubernetes.io/projected/742cb1c3-7009-4642-bc2c-f99378b5d102-kube-api-access-djjdg\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:10.154434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.154354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:10.400869 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.400834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/742cb1c3-7009-4642-bc2c-f99378b5d102-tls-certs\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:10.403410 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.403379 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/742cb1c3-7009-4642-bc2c-f99378b5d102-tls-certs\") pod \"model-serving-api-54d89bd5fc-4gtxj\" (UID: \"742cb1c3-7009-4642-bc2c-f99378b5d102\") " pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:10.428489 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.428465 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hrrwn"] Sep 06 19:34:10.430514 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:34:10.430481 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cee458_63cf_48ee_a6b4_ee109ef946ed.slice/crio-dbf36b4717fe203f0e10e2fc66d37da9b3f87844db9b909e822803f35c54e672 WatchSource:0}: Error finding container dbf36b4717fe203f0e10e2fc66d37da9b3f87844db9b909e822803f35c54e672: Status 404 returned error can't find the container with id dbf36b4717fe203f0e10e2fc66d37da9b3f87844db9b909e822803f35c54e672 Sep 06 19:34:10.555827 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.555788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" event={"ID":"e6cee458-63cf-48ee-a6b4-ee109ef946ed","Type":"ContainerStarted","Data":"dbf36b4717fe203f0e10e2fc66d37da9b3f87844db9b909e822803f35c54e672"} Sep 06 19:34:10.663123 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.663084 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:10.850441 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.850392 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth"] Sep 06 19:34:10.858457 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.858426 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.861870 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.861842 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 06 19:34:10.862060 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.861920 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dw64s\"" Sep 06 19:34:10.864191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.864167 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth"] Sep 06 19:34:10.905700 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905662 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693305e5-c0c7-4161-a927-f0858b17e555-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.905869 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905713 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.905869 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693305e5-c0c7-4161-a927-f0858b17e555-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.905869 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905788 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.905869 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.905869 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905861 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693305e5-c0c7-4161-a927-f0858b17e555-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.906031 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.906031 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905947 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.906031 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.905968 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7vfp\" (UniqueName: \"kubernetes.io/projected/693305e5-c0c7-4161-a927-f0858b17e555-kube-api-access-q7vfp\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:10.934807 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:10.934783 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4gtxj"] Sep 06 19:34:10.937278 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:34:10.937239 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742cb1c3_7009_4642_bc2c_f99378b5d102.slice/crio-38ff2c9b43d0c04734a15e872459083b98a1f0408f6f122ce55793ab40b3ba93 WatchSource:0}: Error finding container 38ff2c9b43d0c04734a15e872459083b98a1f0408f6f122ce55793ab40b3ba93: Status 404 returned error can't find the container with id 38ff2c9b43d0c04734a15e872459083b98a1f0408f6f122ce55793ab40b3ba93 Sep 06 19:34:11.006605 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006515 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006605 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006551 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006605 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006574 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693305e5-c0c7-4161-a927-f0858b17e555-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006707 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7vfp\" (UniqueName: \"kubernetes.io/projected/693305e5-c0c7-4161-a927-f0858b17e555-kube-api-access-q7vfp\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006830 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693305e5-c0c7-4161-a927-f0858b17e555-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.006885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.007141 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.006907 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693305e5-c0c7-4161-a927-f0858b17e555-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.007141 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.007024 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.007141 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.007034 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.007141 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.007113 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.007355 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.007330 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.007541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.007523 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693305e5-c0c7-4161-a927-f0858b17e555-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.009124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.009106 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693305e5-c0c7-4161-a927-f0858b17e555-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.009371 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.009353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693305e5-c0c7-4161-a927-f0858b17e555-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.013973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.013948 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693305e5-c0c7-4161-a927-f0858b17e555-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.014297 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.014280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7vfp\" (UniqueName: \"kubernetes.io/projected/693305e5-c0c7-4161-a927-f0858b17e555-kube-api-access-q7vfp\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2bpth\" (UID: \"693305e5-c0c7-4161-a927-f0858b17e555\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.173298 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.173250 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:11.466130 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.466104 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth"] Sep 06 19:34:11.469784 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:34:11.469395 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693305e5_c0c7_4161_a927_f0858b17e555.slice/crio-f76de7d9662707a783e07681fc479ec62a75c8e62936a8a5cdc0819d94f64064 WatchSource:0}: Error finding container f76de7d9662707a783e07681fc479ec62a75c8e62936a8a5cdc0819d94f64064: Status 404 returned error can't find the container with id f76de7d9662707a783e07681fc479ec62a75c8e62936a8a5cdc0819d94f64064 Sep 06 19:34:11.565361 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.565309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" event={"ID":"693305e5-c0c7-4161-a927-f0858b17e555","Type":"ContainerStarted","Data":"f76de7d9662707a783e07681fc479ec62a75c8e62936a8a5cdc0819d94f64064"} Sep 06 19:34:11.567035 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:11.567003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" event={"ID":"742cb1c3-7009-4642-bc2c-f99378b5d102","Type":"ContainerStarted","Data":"38ff2c9b43d0c04734a15e872459083b98a1f0408f6f122ce55793ab40b3ba93"} Sep 06 19:34:15.589379 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.589332 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" event={"ID":"742cb1c3-7009-4642-bc2c-f99378b5d102","Type":"ContainerStarted","Data":"de4278c8428082085f6eb19b87217454c4f913a0f5ce4e245e1f4418acbae938"} Sep 06 19:34:15.589897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.589478 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:15.591280 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.591254 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" event={"ID":"e6cee458-63cf-48ee-a6b4-ee109ef946ed","Type":"ContainerStarted","Data":"682909c745bd3f150f90c3c9ec6aa5715015b7e0692f7f2e37f17d9ac023bd45"} Sep 06 19:34:15.591445 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.591414 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:15.609594 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.609536 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" podStartSLOduration=2.986614429 podStartE2EDuration="6.609520115s" totalImagesPullingTime="3.622905686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:34:15.608258622 +0000 UTC m=+398.342863352" watchObservedRunningTime="2026-09-06 19:34:15.609520115 +0000 UTC m=+398.344124847" Sep 06 19:34:15.625283 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.625226 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" podStartSLOduration=2.534532641 podStartE2EDuration="6.625205678s" totalImagesPullingTime="4.090673037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:34:15.62386961 +0000 UTC m=+398.358474328" watchObservedRunningTime="2026-09-06 19:34:15.625205678 +0000 UTC m=+398.359810410" Sep 06 19:34:15.768094 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.768063 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:15.768419 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.768394 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:15.773424 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.773400 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:15.926166 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:15.926124 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v"] Sep 06 19:34:16.030002 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.029969 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v"] Sep 06 19:34:16.030155 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.030083 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.033338 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.033300 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 06 19:34:16.033457 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.033441 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 06 19:34:16.033872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.033854 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 06 19:34:16.033937 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.033910 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-t2j56\"" Sep 06 19:34:16.159717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159683 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.159904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.159904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159771 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.159904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159788 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.159904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159860 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk9t5\" (UniqueName: \"kubernetes.io/projected/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-kube-api-access-dk9t5\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.160044 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.160044 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.159975 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261217 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261282 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk9t5\" (UniqueName: \"kubernetes.io/projected/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-kube-api-access-dk9t5\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261478 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.261587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.261516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.262517 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.262450 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.264129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.264098 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.264250 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.264227 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.264607 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.264585 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.264694 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.264676 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.269298 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.269280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.269727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.269709 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk9t5\" (UniqueName: \"kubernetes.io/projected/53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d-kube-api-access-dk9t5\") pod \"istiod-openshift-gateway-5677ccb49b-w2z8v\" (UID: \"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.339832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.339793 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:16.549844 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.549783 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:34:16.549949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.549879 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:34:16.549949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.549909 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:34:16.603190 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.603118 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bc997b6c8-mz6b6" Sep 06 19:34:16.649370 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.649342 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7995f84697-sj4dh"] Sep 06 19:34:16.833476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:16.833451 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v"] Sep 06 19:34:16.834713 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:34:16.834682 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e3dc65_c3d6_4f3c_a71b_6c984cfe6c5d.slice/crio-e3b405ad5ff49a9d57788f57ac25c517f74f4be97570f96f51dd526f9f001570 WatchSource:0}: Error finding container e3b405ad5ff49a9d57788f57ac25c517f74f4be97570f96f51dd526f9f001570: Status 404 returned error can't find the container with id e3b405ad5ff49a9d57788f57ac25c517f74f4be97570f96f51dd526f9f001570 Sep 06 19:34:17.606148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:17.606112 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" event={"ID":"693305e5-c0c7-4161-a927-f0858b17e555","Type":"ContainerStarted","Data":"539238d9324e52339491523ba4f9f05f4f0ce6408066f04ba0be0b461a5d4982"} Sep 06 19:34:17.608329 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:17.608303 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" event={"ID":"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d","Type":"ContainerStarted","Data":"e3b405ad5ff49a9d57788f57ac25c517f74f4be97570f96f51dd526f9f001570"} Sep 06 19:34:17.636631 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:17.636576 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" podStartSLOduration=2.5587136360000002 podStartE2EDuration="7.636560141s" totalImagesPullingTime="5.077846505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:34:17.632650707 +0000 UTC m=+400.367255474" watchObservedRunningTime="2026-09-06 19:34:17.636560141 +0000 UTC m=+400.371164872" Sep 06 19:34:18.173456 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:18.173420 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:18.178820 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:18.178791 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:18.615017 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:18.614938 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:18.616073 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:18.616048 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2bpth" Sep 06 19:34:19.327529 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:19.327482 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:34:19.327604 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:19.327560 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:34:19.623230 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:19.623137 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" event={"ID":"53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d","Type":"ContainerStarted","Data":"17492d9b0166b69c12186db4b4096175952ad8f7173bc8a3fa6249d00980b1b8"} Sep 06 19:34:19.659991 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:19.659926 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" podStartSLOduration=2.170556614 podStartE2EDuration="4.659902648s" totalImagesPullingTime="2.489346034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:34:19.655799055 +0000 UTC m=+402.390403787" watchObservedRunningTime="2026-09-06 19:34:19.659902648 +0000 UTC m=+402.394507383" Sep 06 19:34:20.627598 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:20.627555 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:20.629630 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:20.629607 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-w2z8v" Sep 06 19:34:26.600262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:26.600229 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hrrwn" Sep 06 19:34:26.602126 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:26.602104 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4gtxj" Sep 06 19:34:41.672911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.672845 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7995f84697-sj4dh" podUID="dde3677f-e3f7-48c8-8f55-1d53a9581e9d" containerName="console" containerID="cri-o://749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2" gracePeriod=15 Sep 06 19:34:41.929466 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.929407 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:34:41.986505 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986471 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-config\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.986669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986539 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-trusted-ca-bundle\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.986669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986562 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-oauth-config\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.986669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986577 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-oauth-serving-cert\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.986669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986607 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-668ll\" (UniqueName: \"kubernetes.io/projected/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-kube-api-access-668ll\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.986669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986637 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-serving-cert\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.986974 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986682 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-service-ca\") pod \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\" (UID: \"dde3677f-e3f7-48c8-8f55-1d53a9581e9d\") " Sep 06 19:34:41.987031 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.986863 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-config" (OuterVolumeSpecName: "console-config") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:34:41.987084 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.987061 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:34:41.987125 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.987069 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:34:41.987252 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.987223 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-service-ca" (OuterVolumeSpecName: "service-ca") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:34:41.988964 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.988943 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:34:41.989097 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.988971 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 19:34:41.989097 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:41.988971 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-kube-api-access-668ll" (OuterVolumeSpecName: "kube-api-access-668ll") pod "dde3677f-e3f7-48c8-8f55-1d53a9581e9d" (UID: "dde3677f-e3f7-48c8-8f55-1d53a9581e9d"). InnerVolumeSpecName "kube-api-access-668ll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:34:42.087546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087517 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-trusted-ca-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.087546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087540 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-oauth-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.087546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087550 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-oauth-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.087786 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087558 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-668ll\" (UniqueName: \"kubernetes.io/projected/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-kube-api-access-668ll\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.087786 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087567 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-serving-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.087786 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087575 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.087786 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.087583 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dde3677f-e3f7-48c8-8f55-1d53a9581e9d-console-config\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:34:42.730579 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.730549 2575 generic.go:356] "Generic (PLEG): container finished" podID="dde3677f-e3f7-48c8-8f55-1d53a9581e9d" containerID="749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2" exitCode=0 Sep 06 19:34:42.731009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.730593 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7995f84697-sj4dh" Sep 06 19:34:42.731009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.730631 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7995f84697-sj4dh" event={"ID":"dde3677f-e3f7-48c8-8f55-1d53a9581e9d","Type":"ContainerDied","Data":"749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2"} Sep 06 19:34:42.731009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.730681 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7995f84697-sj4dh" event={"ID":"dde3677f-e3f7-48c8-8f55-1d53a9581e9d","Type":"ContainerDied","Data":"f5002ea0d0c9f85559619967335c1c471e456b601a4ea450065462f35e9bfaf3"} Sep 06 19:34:42.731009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.730702 2575 scope.go:117] "RemoveContainer" containerID="749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2" Sep 06 19:34:42.739698 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.739679 2575 scope.go:117] "RemoveContainer" containerID="749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2" Sep 06 19:34:42.739991 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:34:42.739971 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2\": container with ID starting with 749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2 not found: ID does not exist" containerID="749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2" Sep 06 19:34:42.740043 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.739998 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2"} err="failed to get container status \"749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2\": rpc error: code = NotFound desc = could not find container \"749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2\": container with ID starting with 749f67dc0e6fa7c2d3a9b88e2da7afdaf57ec4cfb699d3ab3093780d4fd2e5a2 not found: ID does not exist" Sep 06 19:34:42.750302 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.750274 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7995f84697-sj4dh"] Sep 06 19:34:42.753510 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:42.753487 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7995f84697-sj4dh"] Sep 06 19:34:43.761711 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:34:43.761675 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde3677f-e3f7-48c8-8f55-1d53a9581e9d" path="/var/lib/kubelet/pods/dde3677f-e3f7-48c8-8f55-1d53a9581e9d/volumes" Sep 06 19:35:13.121421 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.121337 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq"] Sep 06 19:35:13.121895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.121713 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde3677f-e3f7-48c8-8f55-1d53a9581e9d" containerName="console" Sep 06 19:35:13.121895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.121726 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde3677f-e3f7-48c8-8f55-1d53a9581e9d" containerName="console" Sep 06 19:35:13.121895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.121814 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde3677f-e3f7-48c8-8f55-1d53a9581e9d" containerName="console" Sep 06 19:35:13.123918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.123891 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.126374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.126346 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6znk5\"" Sep 06 19:35:13.126485 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.126346 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 19:35:13.127953 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.127935 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 19:35:13.133330 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.133307 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq"] Sep 06 19:35:13.225124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.225088 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6"] Sep 06 19:35:13.227682 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.227664 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.235495 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.235468 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6"] Sep 06 19:35:13.260908 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.260871 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.261081 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.260943 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szzjx\" (UniqueName: \"kubernetes.io/projected/608342fe-f1c1-4188-8919-f1a2746c50e8-kube-api-access-szzjx\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.261081 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.260984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.319311 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.319279 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d"] Sep 06 19:35:13.321779 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.321741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.329788 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.329730 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d"] Sep 06 19:35:13.362123 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szzjx\" (UniqueName: \"kubernetes.io/projected/608342fe-f1c1-4188-8919-f1a2746c50e8-kube-api-access-szzjx\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.362290 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362141 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.362290 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362176 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.362290 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362202 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.362410 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362291 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ljrj\" (UniqueName: \"kubernetes.io/projected/42717207-6e29-4538-a051-44131dfbce26-kube-api-access-6ljrj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.362410 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.362578 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362559 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.362618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.362591 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.369881 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.369860 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szzjx\" (UniqueName: \"kubernetes.io/projected/608342fe-f1c1-4188-8919-f1a2746c50e8-kube-api-access-szzjx\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.423098 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.423059 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc"] Sep 06 19:35:13.425871 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.425850 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.434110 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.434057 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:13.435334 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.435307 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc"] Sep 06 19:35:13.463454 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463424 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.463613 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tln6s\" (UniqueName: \"kubernetes.io/projected/bf16491c-7af3-44cc-a064-0a5844da9b38-kube-api-access-tln6s\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.463613 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463496 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.463613 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463542 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.463613 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463581 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.463781 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463621 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ljrj\" (UniqueName: \"kubernetes.io/projected/42717207-6e29-4538-a051-44131dfbce26-kube-api-access-6ljrj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.463907 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463888 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.463947 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.463917 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.471665 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.471645 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ljrj\" (UniqueName: \"kubernetes.io/projected/42717207-6e29-4538-a051-44131dfbce26-kube-api-access-6ljrj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.537376 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.537341 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:13.564643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564608 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.564854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.564854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.564998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564879 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.564998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564929 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tln6s\" (UniqueName: \"kubernetes.io/projected/bf16491c-7af3-44cc-a064-0a5844da9b38-kube-api-access-tln6s\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.564998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564959 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvf5\" (UniqueName: \"kubernetes.io/projected/43e03c4d-9087-471a-9070-49e2ca685c4f-kube-api-access-2qvf5\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.564998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.564987 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.565338 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.565304 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.573794 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.573768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tln6s\" (UniqueName: \"kubernetes.io/projected/bf16491c-7af3-44cc-a064-0a5844da9b38-kube-api-access-tln6s\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.632723 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.632689 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:13.665895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.665865 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.666050 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.665910 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.666050 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.665971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvf5\" (UniqueName: \"kubernetes.io/projected/43e03c4d-9087-471a-9070-49e2ca685c4f-kube-api-access-2qvf5\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.666284 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.666259 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.666350 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.666284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.674585 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.674526 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvf5\" (UniqueName: \"kubernetes.io/projected/43e03c4d-9087-471a-9070-49e2ca685c4f-kube-api-access-2qvf5\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.709897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.709864 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq"] Sep 06 19:35:13.710664 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:35:13.710629 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608342fe_f1c1_4188_8919_f1a2746c50e8.slice/crio-64e8d5521b571d3854d0cb14af21d80c04ea681c60a5f53e020738c202695258 WatchSource:0}: Error finding container 64e8d5521b571d3854d0cb14af21d80c04ea681c60a5f53e020738c202695258: Status 404 returned error can't find the container with id 64e8d5521b571d3854d0cb14af21d80c04ea681c60a5f53e020738c202695258 Sep 06 19:35:13.737618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.737598 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:13.812098 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.812069 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6"] Sep 06 19:35:13.814205 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:35:13.814161 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42717207_6e29_4538_a051_44131dfbce26.slice/crio-a99e094f1245dc66147cb571c70f2c8e665b08b8b14cf1a99417eb2bc7bf4747 WatchSource:0}: Error finding container a99e094f1245dc66147cb571c70f2c8e665b08b8b14cf1a99417eb2bc7bf4747: Status 404 returned error can't find the container with id a99e094f1245dc66147cb571c70f2c8e665b08b8b14cf1a99417eb2bc7bf4747 Sep 06 19:35:13.881627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.881595 2575 generic.go:356] "Generic (PLEG): container finished" podID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerID="bf3404635f291172eb9712b6d380cb6a66a7db3c7bad91a0ebf614e7d00d59b3" exitCode=0 Sep 06 19:35:13.881770 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.881686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" event={"ID":"608342fe-f1c1-4188-8919-f1a2746c50e8","Type":"ContainerDied","Data":"bf3404635f291172eb9712b6d380cb6a66a7db3c7bad91a0ebf614e7d00d59b3"} Sep 06 19:35:13.881834 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.881770 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" event={"ID":"608342fe-f1c1-4188-8919-f1a2746c50e8","Type":"ContainerStarted","Data":"64e8d5521b571d3854d0cb14af21d80c04ea681c60a5f53e020738c202695258"} Sep 06 19:35:13.883318 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.883246 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" event={"ID":"42717207-6e29-4538-a051-44131dfbce26","Type":"ContainerStarted","Data":"a99e094f1245dc66147cb571c70f2c8e665b08b8b14cf1a99417eb2bc7bf4747"} Sep 06 19:35:13.905085 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:13.905048 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d"] Sep 06 19:35:13.906733 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:35:13.906709 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf16491c_7af3_44cc_a064_0a5844da9b38.slice/crio-f84939ebe25a5e0bfe239d877ca6b4c4e050f4ea7787abdca3aa2f0d6c9355b2 WatchSource:0}: Error finding container f84939ebe25a5e0bfe239d877ca6b4c4e050f4ea7787abdca3aa2f0d6c9355b2: Status 404 returned error can't find the container with id f84939ebe25a5e0bfe239d877ca6b4c4e050f4ea7787abdca3aa2f0d6c9355b2 Sep 06 19:35:14.012910 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.012884 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc"] Sep 06 19:35:14.032528 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:35:14.032499 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e03c4d_9087_471a_9070_49e2ca685c4f.slice/crio-c1e40e60f6957d0d5969c02d983ed7c0a42decae63f62d6d03d433eb8e260a44 WatchSource:0}: Error finding container c1e40e60f6957d0d5969c02d983ed7c0a42decae63f62d6d03d433eb8e260a44: Status 404 returned error can't find the container with id c1e40e60f6957d0d5969c02d983ed7c0a42decae63f62d6d03d433eb8e260a44 Sep 06 19:35:14.891808 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.891733 2575 generic.go:356] "Generic (PLEG): container finished" podID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerID="64e1ac5f1ef87eed037bb4527cf5b0117d1af65e177ac17821039aeb2dd03045" exitCode=0 Sep 06 19:35:14.891808 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.891789 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" event={"ID":"43e03c4d-9087-471a-9070-49e2ca685c4f","Type":"ContainerDied","Data":"64e1ac5f1ef87eed037bb4527cf5b0117d1af65e177ac17821039aeb2dd03045"} Sep 06 19:35:14.892324 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.891889 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" event={"ID":"43e03c4d-9087-471a-9070-49e2ca685c4f","Type":"ContainerStarted","Data":"c1e40e60f6957d0d5969c02d983ed7c0a42decae63f62d6d03d433eb8e260a44"} Sep 06 19:35:14.893492 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.893476 2575 generic.go:356] "Generic (PLEG): container finished" podID="42717207-6e29-4538-a051-44131dfbce26" containerID="b0c46a61c8c754a4336649b2d2da7795563177d07ab1e4fc987d90ba14d5efec" exitCode=0 Sep 06 19:35:14.893562 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.893510 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" event={"ID":"42717207-6e29-4538-a051-44131dfbce26","Type":"ContainerDied","Data":"b0c46a61c8c754a4336649b2d2da7795563177d07ab1e4fc987d90ba14d5efec"} Sep 06 19:35:14.895365 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.895346 2575 generic.go:356] "Generic (PLEG): container finished" podID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerID="4f48160ee9dd97317e548a759edec6c9a102036f93e5a7ad019d68b909d7e3c0" exitCode=0 Sep 06 19:35:14.895461 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.895406 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" event={"ID":"bf16491c-7af3-44cc-a064-0a5844da9b38","Type":"ContainerDied","Data":"4f48160ee9dd97317e548a759edec6c9a102036f93e5a7ad019d68b909d7e3c0"} Sep 06 19:35:14.895461 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:14.895447 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" event={"ID":"bf16491c-7af3-44cc-a064-0a5844da9b38","Type":"ContainerStarted","Data":"f84939ebe25a5e0bfe239d877ca6b4c4e050f4ea7787abdca3aa2f0d6c9355b2"} Sep 06 19:35:15.905386 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:15.905352 2575 generic.go:356] "Generic (PLEG): container finished" podID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerID="0544a15cba0300afec3ffe3da7b5781bc5562a20c9e21f2da3f8fc496d219c73" exitCode=0 Sep 06 19:35:15.905827 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:15.905417 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" event={"ID":"43e03c4d-9087-471a-9070-49e2ca685c4f","Type":"ContainerDied","Data":"0544a15cba0300afec3ffe3da7b5781bc5562a20c9e21f2da3f8fc496d219c73"} Sep 06 19:35:15.907686 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:15.907662 2575 generic.go:356] "Generic (PLEG): container finished" podID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerID="6d2699f1abbdd2d14f5e20185fd3729c020ded5d9e8da8f50d1e7a077013ec51" exitCode=0 Sep 06 19:35:15.907806 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:15.907728 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" event={"ID":"608342fe-f1c1-4188-8919-f1a2746c50e8","Type":"ContainerDied","Data":"6d2699f1abbdd2d14f5e20185fd3729c020ded5d9e8da8f50d1e7a077013ec51"} Sep 06 19:35:15.910029 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:15.910004 2575 generic.go:356] "Generic (PLEG): container finished" podID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerID="95994bc59088555b2d070ad3a6ed59cd35000d83e2212043dcf190791e4d4851" exitCode=0 Sep 06 19:35:15.910144 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:15.910069 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" event={"ID":"bf16491c-7af3-44cc-a064-0a5844da9b38","Type":"ContainerDied","Data":"95994bc59088555b2d070ad3a6ed59cd35000d83e2212043dcf190791e4d4851"} Sep 06 19:35:16.919238 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.919204 2575 generic.go:356] "Generic (PLEG): container finished" podID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerID="85f566256f6443c7aa9df1fbba9463b268969b2d24236098c91c2072d50fa075" exitCode=0 Sep 06 19:35:16.919612 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.919290 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" event={"ID":"bf16491c-7af3-44cc-a064-0a5844da9b38","Type":"ContainerDied","Data":"85f566256f6443c7aa9df1fbba9463b268969b2d24236098c91c2072d50fa075"} Sep 06 19:35:16.921486 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.921463 2575 generic.go:356] "Generic (PLEG): container finished" podID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerID="a2bfcfa5d05d5248e147a735b68910fe1fc8432148347ead84e19fefe14ec569" exitCode=0 Sep 06 19:35:16.921601 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.921541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" event={"ID":"43e03c4d-9087-471a-9070-49e2ca685c4f","Type":"ContainerDied","Data":"a2bfcfa5d05d5248e147a735b68910fe1fc8432148347ead84e19fefe14ec569"} Sep 06 19:35:16.923379 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.923360 2575 generic.go:356] "Generic (PLEG): container finished" podID="42717207-6e29-4538-a051-44131dfbce26" containerID="597c49fa7f0f528ba9f932d8fc02d21cf41a459c7a4097887906560b68215252" exitCode=0 Sep 06 19:35:16.923505 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.923427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" event={"ID":"42717207-6e29-4538-a051-44131dfbce26","Type":"ContainerDied","Data":"597c49fa7f0f528ba9f932d8fc02d21cf41a459c7a4097887906560b68215252"} Sep 06 19:35:16.925729 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.925710 2575 generic.go:356] "Generic (PLEG): container finished" podID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerID="9e1a40375964443cf8d6ecabb43b844d2b6ec383c1a9d7a1722b66aa33f51493" exitCode=0 Sep 06 19:35:16.925827 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:16.925737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" event={"ID":"608342fe-f1c1-4188-8919-f1a2746c50e8","Type":"ContainerDied","Data":"9e1a40375964443cf8d6ecabb43b844d2b6ec383c1a9d7a1722b66aa33f51493"} Sep 06 19:35:17.933627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:17.933597 2575 generic.go:356] "Generic (PLEG): container finished" podID="42717207-6e29-4538-a051-44131dfbce26" containerID="ca6dd718c36eeb3dccd5f9250a5069fc93b674eb0adf1049af7e72798d85487e" exitCode=0 Sep 06 19:35:17.934054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:17.933668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" event={"ID":"42717207-6e29-4538-a051-44131dfbce26","Type":"ContainerDied","Data":"ca6dd718c36eeb3dccd5f9250a5069fc93b674eb0adf1049af7e72798d85487e"} Sep 06 19:35:18.102712 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.102687 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:18.105943 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.105918 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:18.109289 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.109270 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:18.202939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.202849 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-bundle\") pod \"43e03c4d-9087-471a-9070-49e2ca685c4f\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " Sep 06 19:35:18.202939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.202895 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-util\") pod \"608342fe-f1c1-4188-8919-f1a2746c50e8\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " Sep 06 19:35:18.202939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.202914 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-bundle\") pod \"bf16491c-7af3-44cc-a064-0a5844da9b38\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " Sep 06 19:35:18.202939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.202942 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-util\") pod \"bf16491c-7af3-44cc-a064-0a5844da9b38\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " Sep 06 19:35:18.203264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.202968 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-bundle\") pod \"608342fe-f1c1-4188-8919-f1a2746c50e8\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " Sep 06 19:35:18.203264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203032 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-util\") pod \"43e03c4d-9087-471a-9070-49e2ca685c4f\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " Sep 06 19:35:18.203264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203093 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tln6s\" (UniqueName: \"kubernetes.io/projected/bf16491c-7af3-44cc-a064-0a5844da9b38-kube-api-access-tln6s\") pod \"bf16491c-7af3-44cc-a064-0a5844da9b38\" (UID: \"bf16491c-7af3-44cc-a064-0a5844da9b38\") " Sep 06 19:35:18.203264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203128 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qvf5\" (UniqueName: \"kubernetes.io/projected/43e03c4d-9087-471a-9070-49e2ca685c4f-kube-api-access-2qvf5\") pod \"43e03c4d-9087-471a-9070-49e2ca685c4f\" (UID: \"43e03c4d-9087-471a-9070-49e2ca685c4f\") " Sep 06 19:35:18.203264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203158 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szzjx\" (UniqueName: \"kubernetes.io/projected/608342fe-f1c1-4188-8919-f1a2746c50e8-kube-api-access-szzjx\") pod \"608342fe-f1c1-4188-8919-f1a2746c50e8\" (UID: \"608342fe-f1c1-4188-8919-f1a2746c50e8\") " Sep 06 19:35:18.204112 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203643 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-bundle" (OuterVolumeSpecName: "bundle") pod "bf16491c-7af3-44cc-a064-0a5844da9b38" (UID: "bf16491c-7af3-44cc-a064-0a5844da9b38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:18.204112 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203649 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-bundle" (OuterVolumeSpecName: "bundle") pod "43e03c4d-9087-471a-9070-49e2ca685c4f" (UID: "43e03c4d-9087-471a-9070-49e2ca685c4f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:18.204112 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.203939 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-bundle" (OuterVolumeSpecName: "bundle") pod "608342fe-f1c1-4188-8919-f1a2746c50e8" (UID: "608342fe-f1c1-4188-8919-f1a2746c50e8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:18.205623 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.205597 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf16491c-7af3-44cc-a064-0a5844da9b38-kube-api-access-tln6s" (OuterVolumeSpecName: "kube-api-access-tln6s") pod "bf16491c-7af3-44cc-a064-0a5844da9b38" (UID: "bf16491c-7af3-44cc-a064-0a5844da9b38"). InnerVolumeSpecName "kube-api-access-tln6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:35:18.205720 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.205672 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43e03c4d-9087-471a-9070-49e2ca685c4f-kube-api-access-2qvf5" (OuterVolumeSpecName: "kube-api-access-2qvf5") pod "43e03c4d-9087-471a-9070-49e2ca685c4f" (UID: "43e03c4d-9087-471a-9070-49e2ca685c4f"). InnerVolumeSpecName "kube-api-access-2qvf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:35:18.205891 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.205875 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/608342fe-f1c1-4188-8919-f1a2746c50e8-kube-api-access-szzjx" (OuterVolumeSpecName: "kube-api-access-szzjx") pod "608342fe-f1c1-4188-8919-f1a2746c50e8" (UID: "608342fe-f1c1-4188-8919-f1a2746c50e8"). InnerVolumeSpecName "kube-api-access-szzjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:35:18.209261 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.209240 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-util" (OuterVolumeSpecName: "util") pod "bf16491c-7af3-44cc-a064-0a5844da9b38" (UID: "bf16491c-7af3-44cc-a064-0a5844da9b38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:18.212491 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.212463 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-util" (OuterVolumeSpecName: "util") pod "43e03c4d-9087-471a-9070-49e2ca685c4f" (UID: "43e03c4d-9087-471a-9070-49e2ca685c4f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:18.212560 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.212547 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-util" (OuterVolumeSpecName: "util") pod "608342fe-f1c1-4188-8919-f1a2746c50e8" (UID: "608342fe-f1c1-4188-8919-f1a2746c50e8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:18.304440 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304408 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304440 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304437 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304440 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304446 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf16491c-7af3-44cc-a064-0a5844da9b38-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304454 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/608342fe-f1c1-4188-8919-f1a2746c50e8-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304461 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304470 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tln6s\" (UniqueName: \"kubernetes.io/projected/bf16491c-7af3-44cc-a064-0a5844da9b38-kube-api-access-tln6s\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304480 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qvf5\" (UniqueName: \"kubernetes.io/projected/43e03c4d-9087-471a-9070-49e2ca685c4f-kube-api-access-2qvf5\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304488 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szzjx\" (UniqueName: \"kubernetes.io/projected/608342fe-f1c1-4188-8919-f1a2746c50e8-kube-api-access-szzjx\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.304644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.304497 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43e03c4d-9087-471a-9070-49e2ca685c4f-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:18.940970 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.940934 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" Sep 06 19:35:18.940970 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.940952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfjglc" event={"ID":"43e03c4d-9087-471a-9070-49e2ca685c4f","Type":"ContainerDied","Data":"c1e40e60f6957d0d5969c02d983ed7c0a42decae63f62d6d03d433eb8e260a44"} Sep 06 19:35:18.941455 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.940996 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1e40e60f6957d0d5969c02d983ed7c0a42decae63f62d6d03d433eb8e260a44" Sep 06 19:35:18.943066 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.943046 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" Sep 06 19:35:18.943202 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.943072 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb89jwq" event={"ID":"608342fe-f1c1-4188-8919-f1a2746c50e8","Type":"ContainerDied","Data":"64e8d5521b571d3854d0cb14af21d80c04ea681c60a5f53e020738c202695258"} Sep 06 19:35:18.943202 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.943112 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64e8d5521b571d3854d0cb14af21d80c04ea681c60a5f53e020738c202695258" Sep 06 19:35:18.945055 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.945035 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" Sep 06 19:35:18.945166 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.945066 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0z6b5d" event={"ID":"bf16491c-7af3-44cc-a064-0a5844da9b38","Type":"ContainerDied","Data":"f84939ebe25a5e0bfe239d877ca6b4c4e050f4ea7787abdca3aa2f0d6c9355b2"} Sep 06 19:35:18.945166 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:18.945106 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f84939ebe25a5e0bfe239d877ca6b4c4e050f4ea7787abdca3aa2f0d6c9355b2" Sep 06 19:35:19.081125 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.081102 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:35:19.212339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.212248 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-bundle\") pod \"42717207-6e29-4538-a051-44131dfbce26\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " Sep 06 19:35:19.212339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.212292 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-util\") pod \"42717207-6e29-4538-a051-44131dfbce26\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " Sep 06 19:35:19.212339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.212316 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ljrj\" (UniqueName: \"kubernetes.io/projected/42717207-6e29-4538-a051-44131dfbce26-kube-api-access-6ljrj\") pod \"42717207-6e29-4538-a051-44131dfbce26\" (UID: \"42717207-6e29-4538-a051-44131dfbce26\") " Sep 06 19:35:19.213050 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.212925 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-bundle" (OuterVolumeSpecName: "bundle") pod "42717207-6e29-4538-a051-44131dfbce26" (UID: "42717207-6e29-4538-a051-44131dfbce26"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:19.214542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.214513 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42717207-6e29-4538-a051-44131dfbce26-kube-api-access-6ljrj" (OuterVolumeSpecName: "kube-api-access-6ljrj") pod "42717207-6e29-4538-a051-44131dfbce26" (UID: "42717207-6e29-4538-a051-44131dfbce26"). InnerVolumeSpecName "kube-api-access-6ljrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:35:19.217764 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.217707 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-util" (OuterVolumeSpecName: "util") pod "42717207-6e29-4538-a051-44131dfbce26" (UID: "42717207-6e29-4538-a051-44131dfbce26"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:35:19.313649 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.313612 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-bundle\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:19.313649 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.313640 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42717207-6e29-4538-a051-44131dfbce26-util\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:19.313649 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.313650 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ljrj\" (UniqueName: \"kubernetes.io/projected/42717207-6e29-4538-a051-44131dfbce26-kube-api-access-6ljrj\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:35:19.954515 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.954484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" event={"ID":"42717207-6e29-4538-a051-44131dfbce26","Type":"ContainerDied","Data":"a99e094f1245dc66147cb571c70f2c8e665b08b8b14cf1a99417eb2bc7bf4747"} Sep 06 19:35:19.954904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.954536 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a99e094f1245dc66147cb571c70f2c8e665b08b8b14cf1a99417eb2bc7bf4747" Sep 06 19:35:19.954904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:35:19.954493 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870535l6t6" Sep 06 19:37:37.675610 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:37.675584 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:37:37.678334 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:37.678314 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:37:48.822557 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822522 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dc6h2"] Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822860 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822873 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822882 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822889 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822896 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822901 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822909 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822914 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822919 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822924 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822930 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822935 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822947 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822952 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822960 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822964 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="pull" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822970 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822975 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822980 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822985 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822991 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.822995 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.823003 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.823009 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="util" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.823057 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="43e03c4d-9087-471a-9070-49e2ca685c4f" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.823066 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="42717207-6e29-4538-a051-44131dfbce26" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.823075 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf16491c-7af3-44cc-a064-0a5844da9b38" containerName="extract" Sep 06 19:37:48.824897 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.823085 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="608342fe-f1c1-4188-8919-f1a2746c50e8" containerName="extract" Sep 06 19:37:48.825872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.825858 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:48.828304 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.828280 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fldd8\"" Sep 06 19:37:48.828411 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.828304 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 06 19:37:48.835402 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.835371 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dc6h2"] Sep 06 19:37:48.921769 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.921712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881dd41f-b5f1-4930-8e55-a219f83580f1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dc6h2\" (UID: \"881dd41f-b5f1-4930-8e55-a219f83580f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:48.921769 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:48.921770 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gpfs\" (UniqueName: \"kubernetes.io/projected/881dd41f-b5f1-4930-8e55-a219f83580f1-kube-api-access-5gpfs\") pod \"payload-processing-5b54b7fbb7-dc6h2\" (UID: \"881dd41f-b5f1-4930-8e55-a219f83580f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:49.022928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.022883 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881dd41f-b5f1-4930-8e55-a219f83580f1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dc6h2\" (UID: \"881dd41f-b5f1-4930-8e55-a219f83580f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:49.023104 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.022926 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gpfs\" (UniqueName: \"kubernetes.io/projected/881dd41f-b5f1-4930-8e55-a219f83580f1-kube-api-access-5gpfs\") pod \"payload-processing-5b54b7fbb7-dc6h2\" (UID: \"881dd41f-b5f1-4930-8e55-a219f83580f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:49.023546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.023524 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881dd41f-b5f1-4930-8e55-a219f83580f1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dc6h2\" (UID: \"881dd41f-b5f1-4930-8e55-a219f83580f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:49.030380 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.030352 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gpfs\" (UniqueName: \"kubernetes.io/projected/881dd41f-b5f1-4930-8e55-a219f83580f1-kube-api-access-5gpfs\") pod \"payload-processing-5b54b7fbb7-dc6h2\" (UID: \"881dd41f-b5f1-4930-8e55-a219f83580f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:49.137728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.137654 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:49.408766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.408639 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dc6h2"] Sep 06 19:37:49.410851 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:37:49.410824 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881dd41f_b5f1_4930_8e55_a219f83580f1.slice/crio-3db8799189a1ae550c4eeaf7105b9ea66759dc1c696390fce4a05012705fe7e7 WatchSource:0}: Error finding container 3db8799189a1ae550c4eeaf7105b9ea66759dc1c696390fce4a05012705fe7e7: Status 404 returned error can't find the container with id 3db8799189a1ae550c4eeaf7105b9ea66759dc1c696390fce4a05012705fe7e7 Sep 06 19:37:49.412839 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.412820 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 19:37:49.711217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:49.711183 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" event={"ID":"881dd41f-b5f1-4930-8e55-a219f83580f1","Type":"ContainerStarted","Data":"3db8799189a1ae550c4eeaf7105b9ea66759dc1c696390fce4a05012705fe7e7"} Sep 06 19:37:51.725246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:51.725212 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" event={"ID":"881dd41f-b5f1-4930-8e55-a219f83580f1","Type":"ContainerStarted","Data":"152e1d157a66352ae126b1e15ce14883c4a3fa2a029ef2b4e61fe74a4bb8c0ba"} Sep 06 19:37:51.725654 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:51.725401 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:37:51.739470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:37:51.739415 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" podStartSLOduration=2.153355109 podStartE2EDuration="3.739371492s" totalImagesPullingTime="1.586016383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:37:51.739332613 +0000 UTC m=+614.473937346" watchObservedRunningTime="2026-09-06 19:37:51.739371492 +0000 UTC m=+614.473976226" Sep 06 19:38:02.732877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:02.732797 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-dc6h2" Sep 06 19:38:13.640154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.639654 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z"] Sep 06 19:38:13.644475 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.644448 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.646911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.646890 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 06 19:38:13.647920 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.647898 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ldj7j\"" Sep 06 19:38:13.647920 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.647916 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 06 19:38:13.648077 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.647919 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 06 19:38:13.653956 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.653934 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z"] Sep 06 19:38:13.741742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.741707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj9qb\" (UniqueName: \"kubernetes.io/projected/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-kube-api-access-xj9qb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.741742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.741759 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.742014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.741824 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.742014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.741891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.742014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.741922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.742014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.741943 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.842639 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.842605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj9qb\" (UniqueName: \"kubernetes.io/projected/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-kube-api-access-xj9qb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.842857 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.842654 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.842857 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.842777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.842857 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.842823 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.843046 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.842858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.843046 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.842904 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.843156 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.843096 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.843214 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.843173 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.843316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.843294 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.845219 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.845198 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.845534 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.845518 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.849984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.849963 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj9qb\" (UniqueName: \"kubernetes.io/projected/ac48f8e5-b322-4c8d-848e-027bf79cbdbc-kube-api-access-xj9qb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mvc6z\" (UID: \"ac48f8e5-b322-4c8d-848e-027bf79cbdbc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:13.957906 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:13.957871 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:14.230550 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:14.230524 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z"] Sep 06 19:38:14.232487 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:38:14.232464 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac48f8e5_b322_4c8d_848e_027bf79cbdbc.slice/crio-24b440d7aaa31e97dbc5aa8997da62d7a54f1594786c4bc663411e7c60523e6d WatchSource:0}: Error finding container 24b440d7aaa31e97dbc5aa8997da62d7a54f1594786c4bc663411e7c60523e6d: Status 404 returned error can't find the container with id 24b440d7aaa31e97dbc5aa8997da62d7a54f1594786c4bc663411e7c60523e6d Sep 06 19:38:14.853741 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:14.853702 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" event={"ID":"ac48f8e5-b322-4c8d-848e-027bf79cbdbc","Type":"ContainerStarted","Data":"24b440d7aaa31e97dbc5aa8997da62d7a54f1594786c4bc663411e7c60523e6d"} Sep 06 19:38:15.861248 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:15.861212 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" event={"ID":"ac48f8e5-b322-4c8d-848e-027bf79cbdbc","Type":"ContainerStarted","Data":"7615777ccb5738d273f2beace18cb38dd29064771a1350837746e4003a6522f7"} Sep 06 19:38:15.878615 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:15.878567 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" podStartSLOduration=1.896859453 podStartE2EDuration="2.878551226s" totalImagesPullingTime="981.691773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:38:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:38:15.876593492 +0000 UTC m=+638.611198237" watchObservedRunningTime="2026-09-06 19:38:15.878551226 +0000 UTC m=+638.613155958" Sep 06 19:38:23.958979 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:23.958937 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:23.959488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:23.958997 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:23.971248 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:23.971224 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:38:24.917292 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:38:24.917252 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mvc6z" Sep 06 19:39:17.541773 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.541715 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7"] Sep 06 19:39:17.545071 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.545055 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.547995 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.547977 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 06 19:39:17.554525 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.554503 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7"] Sep 06 19:39:17.620902 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.620869 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.621087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.620926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l55dj\" (UniqueName: \"kubernetes.io/projected/c2278452-df71-4eaa-bf5c-162e8f5e2025-kube-api-access-l55dj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.621087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.620953 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.621087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.621012 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2278452-df71-4eaa-bf5c-162e8f5e2025-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.621087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.621059 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.621272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.621093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722255 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722212 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722445 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722277 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l55dj\" (UniqueName: \"kubernetes.io/projected/c2278452-df71-4eaa-bf5c-162e8f5e2025-kube-api-access-l55dj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722445 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722303 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722445 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722370 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2278452-df71-4eaa-bf5c-162e8f5e2025-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722445 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722870 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722836 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.722870 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.722861 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.723058 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.723039 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.725016 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.724993 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2278452-df71-4eaa-bf5c-162e8f5e2025-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.725174 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.725151 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2278452-df71-4eaa-bf5c-162e8f5e2025-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.729842 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.729810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l55dj\" (UniqueName: \"kubernetes.io/projected/c2278452-df71-4eaa-bf5c-162e8f5e2025-kube-api-access-l55dj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9jzp7\" (UID: \"c2278452-df71-4eaa-bf5c-162e8f5e2025\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:17.857302 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:17.857184 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:18.134339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:18.134314 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7"] Sep 06 19:39:18.136189 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:39:18.136159 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2278452_df71_4eaa_bf5c_162e8f5e2025.slice/crio-63518dfe94118f4ac7a505cedf931303b2bfdc724d029b6babbf681a3de9d164 WatchSource:0}: Error finding container 63518dfe94118f4ac7a505cedf931303b2bfdc724d029b6babbf681a3de9d164: Status 404 returned error can't find the container with id 63518dfe94118f4ac7a505cedf931303b2bfdc724d029b6babbf681a3de9d164 Sep 06 19:39:18.190819 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:18.190779 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" event={"ID":"c2278452-df71-4eaa-bf5c-162e8f5e2025","Type":"ContainerStarted","Data":"63518dfe94118f4ac7a505cedf931303b2bfdc724d029b6babbf681a3de9d164"} Sep 06 19:39:19.197448 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:19.197400 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" event={"ID":"c2278452-df71-4eaa-bf5c-162e8f5e2025","Type":"ContainerStarted","Data":"5dc06b7978c619fec15baa2c41319c4a5f5f5862a29469fe9b3bc307d38aea49"} Sep 06 19:39:19.232831 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:19.232782 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" podStartSLOduration=2.054336476 podStartE2EDuration="2.232737506s" totalImagesPullingTime="178.40103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:39:19.231367075 +0000 UTC m=+701.965971811" watchObservedRunningTime="2026-09-06 19:39:19.232737506 +0000 UTC m=+701.967342237" Sep 06 19:39:27.857824 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:27.857786 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:27.858232 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:27.857845 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:27.872399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:27.872374 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:39:28.260977 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:39:28.260941 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9jzp7" Sep 06 19:41:20.621389 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.621353 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq"] Sep 06 19:41:20.625545 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.625524 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.628072 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.628047 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6433a6-openshift-default-dockercfg-t67kc\"" Sep 06 19:41:20.635020 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.634998 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq"] Sep 06 19:41:20.783526 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-credential-socket\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783526 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783528 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-data\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istiod-ca-cert\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783590 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh4z4\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-kube-api-access-gh4z4\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-token\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-podinfo\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-certs\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-envoy\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.783922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.783831 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-socket\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.884742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884659 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-credential-socket\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.884742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884697 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-data\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.884742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884715 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istiod-ca-cert\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884772 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh4z4\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-kube-api-access-gh4z4\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884792 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-token\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-podinfo\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884880 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-certs\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884896 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-envoy\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.884914 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-socket\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.885167 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-credential-socket\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.885291 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-certs\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.885314 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-socket\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885436 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.885398 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-data\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.885559 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.885537 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istiod-ca-cert\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.887351 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.887325 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-envoy\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.887575 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.887556 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-podinfo\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.892165 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.892143 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-token\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.892340 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.892320 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh4z4\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-kube-api-access-gh4z4\") pod \"e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:20.938978 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:20.938938 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:21.222648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.222601 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq"] Sep 06 19:41:21.229121 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:21.229093 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd25204_c6e7_43ac_a006_63c40de8f0d7.slice/crio-812984d18a7967cdd65f9b7dfdebc13582cd1fc8ba20d5b17aae412d491c0abb WatchSource:0}: Error finding container 812984d18a7967cdd65f9b7dfdebc13582cd1fc8ba20d5b17aae412d491c0abb: Status 404 returned error can't find the container with id 812984d18a7967cdd65f9b7dfdebc13582cd1fc8ba20d5b17aae412d491c0abb Sep 06 19:41:21.231867 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.231824 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:41:21.231966 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.231948 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:41:21.232014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.231992 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:41:21.839970 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.839935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" event={"ID":"9bd25204-c6e7-43ac-a006-63c40de8f0d7","Type":"ContainerStarted","Data":"2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a"} Sep 06 19:41:21.840399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.839988 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" event={"ID":"9bd25204-c6e7-43ac-a006-63c40de8f0d7","Type":"ContainerStarted","Data":"812984d18a7967cdd65f9b7dfdebc13582cd1fc8ba20d5b17aae412d491c0abb"} Sep 06 19:41:21.874284 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.874229 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podStartSLOduration=1.8742085579999999 podStartE2EDuration="1.874208558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:21.872675497 +0000 UTC m=+824.607280225" watchObservedRunningTime="2026-09-06 19:41:21.874208558 +0000 UTC m=+824.608813290" Sep 06 19:41:21.939646 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:21.939618 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:22.945135 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:22.945103 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:23.850460 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:23.850429 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:23.851481 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:23.851465 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:25.772576 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.772549 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh"] Sep 06 19:41:25.775920 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.775901 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:25.778246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.778228 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6433a6\"" Sep 06 19:41:25.778409 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.778390 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6433a6-dockercfg-cmpvz\"" Sep 06 19:41:25.786853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.786830 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh"] Sep 06 19:41:25.933623 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.933579 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shhmb\" (UniqueName: \"kubernetes.io/projected/3311e9b7-16ec-45b2-9b6e-b20481820681-kube-api-access-shhmb\") pod \"payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:25.933808 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:25.933636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3311e9b7-16ec-45b2-9b6e-b20481820681-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.034890 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.034797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shhmb\" (UniqueName: \"kubernetes.io/projected/3311e9b7-16ec-45b2-9b6e-b20481820681-kube-api-access-shhmb\") pod \"payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.034890 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.034862 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3311e9b7-16ec-45b2-9b6e-b20481820681-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.035558 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.035537 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3311e9b7-16ec-45b2-9b6e-b20481820681-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.042097 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.042070 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shhmb\" (UniqueName: \"kubernetes.io/projected/3311e9b7-16ec-45b2-9b6e-b20481820681-kube-api-access-shhmb\") pod \"payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.088116 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.088084 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.368400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.368355 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh"] Sep 06 19:41:26.369635 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:26.369611 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3311e9b7_16ec_45b2_9b6e_b20481820681.slice/crio-dea3ff6cdce8216f890339ef182a9bec2feee6d1190ccdf4f0ee9761f1b18fb7 WatchSource:0}: Error finding container dea3ff6cdce8216f890339ef182a9bec2feee6d1190ccdf4f0ee9761f1b18fb7: Status 404 returned error can't find the container with id dea3ff6cdce8216f890339ef182a9bec2feee6d1190ccdf4f0ee9761f1b18fb7 Sep 06 19:41:26.867717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.867669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" event={"ID":"3311e9b7-16ec-45b2-9b6e-b20481820681","Type":"ContainerStarted","Data":"8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6"} Sep 06 19:41:26.868119 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.867733 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" event={"ID":"3311e9b7-16ec-45b2-9b6e-b20481820681","Type":"ContainerStarted","Data":"dea3ff6cdce8216f890339ef182a9bec2feee6d1190ccdf4f0ee9761f1b18fb7"} Sep 06 19:41:26.868119 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.867873 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:26.886729 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:26.886673 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" podStartSLOduration=1.886656194 podStartE2EDuration="1.886656194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:26.883573331 +0000 UTC m=+829.618178064" watchObservedRunningTime="2026-09-06 19:41:26.886656194 +0000 UTC m=+829.621260926" Sep 06 19:41:32.990714 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:32.990678 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9"] Sep 06 19:41:33.017772 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.017030 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9"] Sep 06 19:41:33.017772 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.017240 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.021207 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.020343 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4e3f8e3b\"" Sep 06 19:41:33.025808 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.025780 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2"] Sep 06 19:41:33.029851 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.029821 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.032398 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.032371 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4e3f8e3b-dockercfg-cpk4d\"" Sep 06 19:41:33.040844 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.040820 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2"] Sep 06 19:41:33.107391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.107344 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4030ff8-ddae-498b-a194-fbad92585aa8-plugins-config-volume\") pod \"payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.107687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.107658 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dvvx\" (UniqueName: \"kubernetes.io/projected/d4030ff8-ddae-498b-a194-fbad92585aa8-kube-api-access-5dvvx\") pod \"payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.107872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.107854 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.108015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.107999 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhtkb\" (UniqueName: \"kubernetes.io/projected/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-kube-api-access-jhtkb\") pod \"payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.209034 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.208993 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4030ff8-ddae-498b-a194-fbad92585aa8-plugins-config-volume\") pod \"payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.210230 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.210208 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dvvx\" (UniqueName: \"kubernetes.io/projected/d4030ff8-ddae-498b-a194-fbad92585aa8-kube-api-access-5dvvx\") pod \"payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.210782 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.210742 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.211825 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.211806 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhtkb\" (UniqueName: \"kubernetes.io/projected/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-kube-api-access-jhtkb\") pod \"payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.212013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.211724 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.212126 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.210140 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4030ff8-ddae-498b-a194-fbad92585aa8-plugins-config-volume\") pod \"payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.219217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.219193 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dvvx\" (UniqueName: \"kubernetes.io/projected/d4030ff8-ddae-498b-a194-fbad92585aa8-kube-api-access-5dvvx\") pod \"payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.219724 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.219703 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhtkb\" (UniqueName: \"kubernetes.io/projected/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-kube-api-access-jhtkb\") pod \"payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.333108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.333031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.344117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.343882 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.634256 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.634172 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9"] Sep 06 19:41:33.637808 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:33.637780 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7c2d49_d326_4238_be96_ca89f1a3fd4a.slice/crio-ce8fb23b8e43f62daddfc1c1596ac5c55c38dd6fc0260bf44bb0da581ad994c8 WatchSource:0}: Error finding container ce8fb23b8e43f62daddfc1c1596ac5c55c38dd6fc0260bf44bb0da581ad994c8: Status 404 returned error can't find the container with id ce8fb23b8e43f62daddfc1c1596ac5c55c38dd6fc0260bf44bb0da581ad994c8 Sep 06 19:41:33.656469 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.656441 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2"] Sep 06 19:41:33.658802 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:33.658683 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4030ff8_ddae_498b_a194_fbad92585aa8.slice/crio-70380fad45436783c68e0e32031eb82aac1fad377da1c330d6cdbca558099e03 WatchSource:0}: Error finding container 70380fad45436783c68e0e32031eb82aac1fad377da1c330d6cdbca558099e03: Status 404 returned error can't find the container with id 70380fad45436783c68e0e32031eb82aac1fad377da1c330d6cdbca558099e03 Sep 06 19:41:33.905998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.905910 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x"] Sep 06 19:41:33.920132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.920099 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:33.924423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.923126 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0ebbc9\"" Sep 06 19:41:33.924423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.923743 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0ebbc9-dockercfg-2bhzc\"" Sep 06 19:41:33.924912 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.924888 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x"] Sep 06 19:41:33.939723 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.938833 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" event={"ID":"d4030ff8-ddae-498b-a194-fbad92585aa8","Type":"ContainerStarted","Data":"66f435d3fc1120e258bb8712723c54309ad3fd94f299bf340f9c346800739f2f"} Sep 06 19:41:33.939723 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.938898 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" event={"ID":"d4030ff8-ddae-498b-a194-fbad92585aa8","Type":"ContainerStarted","Data":"70380fad45436783c68e0e32031eb82aac1fad377da1c330d6cdbca558099e03"} Sep 06 19:41:33.939723 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.939655 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:33.942262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.942159 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" event={"ID":"1b7c2d49-d326-4238-be96-ca89f1a3fd4a","Type":"ContainerStarted","Data":"851eff8e1e646288ef00823887e7c94b2bf54214d11e94649b74cdbc45823d08"} Sep 06 19:41:33.942262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.942203 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" event={"ID":"1b7c2d49-d326-4238-be96-ca89f1a3fd4a","Type":"ContainerStarted","Data":"ce8fb23b8e43f62daddfc1c1596ac5c55c38dd6fc0260bf44bb0da581ad994c8"} Sep 06 19:41:33.942262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.942231 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:33.958708 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.958657 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" podStartSLOduration=0.958642633 podStartE2EDuration="958.642633ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:33.956601494 +0000 UTC m=+836.691206227" watchObservedRunningTime="2026-09-06 19:41:33.958642633 +0000 UTC m=+836.693247365" Sep 06 19:41:33.976000 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:33.975955 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" podStartSLOduration=1.975939638 podStartE2EDuration="1.975939638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:33.974693736 +0000 UTC m=+836.709298461" watchObservedRunningTime="2026-09-06 19:41:33.975939638 +0000 UTC m=+836.710544369" Sep 06 19:41:34.026348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.026310 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvrm4\" (UniqueName: \"kubernetes.io/projected/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-kube-api-access-qvrm4\") pod \"payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.026348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.026353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.127248 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.127207 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvrm4\" (UniqueName: \"kubernetes.io/projected/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-kube-api-access-qvrm4\") pod \"payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.127521 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.127502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.128583 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.128551 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.136928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.136896 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvrm4\" (UniqueName: \"kubernetes.io/projected/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-kube-api-access-qvrm4\") pod \"payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.248619 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.248588 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:34.554393 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.554370 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x"] Sep 06 19:41:34.556493 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:34.556463 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe8e164_e1fb_4ade_b5e4_69cbfbababa0.slice/crio-3fd7256e54883a249bbbf5c6036c16f92989e26e5edf9c7886fdca4c92065fbc WatchSource:0}: Error finding container 3fd7256e54883a249bbbf5c6036c16f92989e26e5edf9c7886fdca4c92065fbc: Status 404 returned error can't find the container with id 3fd7256e54883a249bbbf5c6036c16f92989e26e5edf9c7886fdca4c92065fbc Sep 06 19:41:34.953742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.953700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" event={"ID":"efe8e164-e1fb-4ade-b5e4-69cbfbababa0","Type":"ContainerStarted","Data":"809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6"} Sep 06 19:41:34.953968 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.953823 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" event={"ID":"efe8e164-e1fb-4ade-b5e4-69cbfbababa0","Type":"ContainerStarted","Data":"3fd7256e54883a249bbbf5c6036c16f92989e26e5edf9c7886fdca4c92065fbc"} Sep 06 19:41:34.970825 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:34.970722 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" podStartSLOduration=1.970706906 podStartE2EDuration="1.970706906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:34.968692031 +0000 UTC m=+837.703296758" watchObservedRunningTime="2026-09-06 19:41:34.970706906 +0000 UTC m=+837.705311638" Sep 06 19:41:35.958400 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:35.958360 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:37.875155 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:37.875119 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:41:39.891922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:39.891879 2575 patch_prober.go:28] interesting pod/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 06 19:41:39.892334 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:39.891954 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 06 19:41:41.997645 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:41.997613 2575 generic.go:356] "Generic (PLEG): container finished" podID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerID="2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a" exitCode=0 Sep 06 19:41:41.998273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:41.997694 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" event={"ID":"9bd25204-c6e7-43ac-a006-63c40de8f0d7","Type":"ContainerDied","Data":"2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a"} Sep 06 19:41:41.998273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:41.998225 2575 scope.go:117] "RemoveContainer" containerID="2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a" Sep 06 19:41:41.999388 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:41.999346 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:41:41.999484 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:41.999435 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:41:41.999484 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:41.999476 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:41:43.010659 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:43.010621 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" event={"ID":"9bd25204-c6e7-43ac-a006-63c40de8f0d7","Type":"ContainerStarted","Data":"f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8"} Sep 06 19:41:43.940074 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:43.940041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:43.940236 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:43.940220 2575 patch_prober.go:28] interesting pod/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 06 19:41:43.940300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:43.940263 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 06 19:41:44.939367 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:44.939328 2575 patch_prober.go:28] interesting pod/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 06 19:41:44.939798 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:44.939398 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 06 19:41:44.956706 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:44.956680 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:41:45.939672 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:45.939631 2575 patch_prober.go:28] interesting pod/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 06 19:41:45.940115 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:45.939700 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 06 19:41:45.960325 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:45.960299 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:41:46.940202 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:46.940165 2575 patch_prober.go:28] interesting pod/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 06 19:41:46.940673 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:46.940237 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 06 19:41:46.967733 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:46.967708 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:41:48.169621 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:48.169581 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:48.170121 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:48.170032 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:48.171202 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:48.171179 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:41:53.567832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.567791 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5"] Sep 06 19:41:53.571349 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.571328 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.573590 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.573552 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d5bb2d\"" Sep 06 19:41:53.573704 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.573609 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d5bb2d-dockercfg-2wrlz\"" Sep 06 19:41:53.583291 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.583244 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5"] Sep 06 19:41:53.612595 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.612560 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9"] Sep 06 19:41:53.615917 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.615893 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.620057 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.620029 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24879275-939e-4705-b536-72f780df3601-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.620191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.620073 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5trl\" (UniqueName: \"kubernetes.io/projected/24879275-939e-4705-b536-72f780df3601-kube-api-access-r5trl\") pod \"payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.627729 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.627700 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9"] Sep 06 19:41:53.720806 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.720776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22bk4\" (UniqueName: \"kubernetes.io/projected/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-kube-api-access-22bk4\") pod \"payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.721013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.720824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24879275-939e-4705-b536-72f780df3601-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.721013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.720887 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5trl\" (UniqueName: \"kubernetes.io/projected/24879275-939e-4705-b536-72f780df3601-kube-api-access-r5trl\") pod \"payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.721013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.721002 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.721461 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.721441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24879275-939e-4705-b536-72f780df3601-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.728996 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.728970 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5trl\" (UniqueName: \"kubernetes.io/projected/24879275-939e-4705-b536-72f780df3601-kube-api-access-r5trl\") pod \"payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.822564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.822460 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.822564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.822521 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22bk4\" (UniqueName: \"kubernetes.io/projected/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-kube-api-access-22bk4\") pod \"payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.823191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.823170 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.830329 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.830298 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22bk4\" (UniqueName: \"kubernetes.io/projected/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-kube-api-access-22bk4\") pod \"payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:53.884535 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.884492 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:53.927562 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:53.927518 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:54.159700 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:54.159666 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5"] Sep 06 19:41:54.161124 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:54.161094 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24879275_939e_4705_b536_72f780df3601.slice/crio-f2834eef568141d737781e9b57742b265f3c28f52299bc205e32155bf5283056 WatchSource:0}: Error finding container f2834eef568141d737781e9b57742b265f3c28f52299bc205e32155bf5283056: Status 404 returned error can't find the container with id f2834eef568141d737781e9b57742b265f3c28f52299bc205e32155bf5283056 Sep 06 19:41:54.211380 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:54.211352 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9"] Sep 06 19:41:54.220997 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:41:54.220970 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17f9a76_43d1_4691_8d0a_17b6e94fd1b9.slice/crio-2f15e19e6ce4f24317227e4410260a84c4edfca976ff40e8491026f869c1986d WatchSource:0}: Error finding container 2f15e19e6ce4f24317227e4410260a84c4edfca976ff40e8491026f869c1986d: Status 404 returned error can't find the container with id 2f15e19e6ce4f24317227e4410260a84c4edfca976ff40e8491026f869c1986d Sep 06 19:41:55.083217 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.083187 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" event={"ID":"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9","Type":"ContainerStarted","Data":"d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc"} Sep 06 19:41:55.083706 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.083245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" event={"ID":"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9","Type":"ContainerStarted","Data":"2f15e19e6ce4f24317227e4410260a84c4edfca976ff40e8491026f869c1986d"} Sep 06 19:41:55.083706 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.083387 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:41:55.085086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.085060 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" event={"ID":"24879275-939e-4705-b536-72f780df3601","Type":"ContainerStarted","Data":"691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3"} Sep 06 19:41:55.085199 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.085105 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" event={"ID":"24879275-939e-4705-b536-72f780df3601","Type":"ContainerStarted","Data":"f2834eef568141d737781e9b57742b265f3c28f52299bc205e32155bf5283056"} Sep 06 19:41:55.085199 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.085131 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:41:55.101410 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.101359 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" podStartSLOduration=2.101340667 podStartE2EDuration="2.101340667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:55.097663655 +0000 UTC m=+857.832268389" watchObservedRunningTime="2026-09-06 19:41:55.101340667 +0000 UTC m=+857.835945395" Sep 06 19:41:55.117565 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:41:55.117506 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" podStartSLOduration=2.117487675 podStartE2EDuration="2.117487675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:41:55.115558839 +0000 UTC m=+857.850163573" watchObservedRunningTime="2026-09-06 19:41:55.117487675 +0000 UTC m=+857.852092408" Sep 06 19:42:02.954851 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:02.954812 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2"] Sep 06 19:42:02.955234 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:02.955035 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" podUID="d4030ff8-ddae-498b-a194-fbad92585aa8" containerName="payload-processing" containerID="cri-o://66f435d3fc1120e258bb8712723c54309ad3fd94f299bf340f9c346800739f2f" gracePeriod=30 Sep 06 19:42:02.966393 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:02.966362 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9"] Sep 06 19:42:02.967021 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:02.966987 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" podUID="1b7c2d49-d326-4238-be96-ca89f1a3fd4a" containerName="payload-pre-processing" containerID="cri-o://851eff8e1e646288ef00823887e7c94b2bf54214d11e94649b74cdbc45823d08" gracePeriod=30 Sep 06 19:42:03.129713 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.129676 2575 generic.go:356] "Generic (PLEG): container finished" podID="1b7c2d49-d326-4238-be96-ca89f1a3fd4a" containerID="851eff8e1e646288ef00823887e7c94b2bf54214d11e94649b74cdbc45823d08" exitCode=0 Sep 06 19:42:03.129891 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.129767 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" event={"ID":"1b7c2d49-d326-4238-be96-ca89f1a3fd4a","Type":"ContainerDied","Data":"851eff8e1e646288ef00823887e7c94b2bf54214d11e94649b74cdbc45823d08"} Sep 06 19:42:03.131718 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.131694 2575 generic.go:356] "Generic (PLEG): container finished" podID="d4030ff8-ddae-498b-a194-fbad92585aa8" containerID="66f435d3fc1120e258bb8712723c54309ad3fd94f299bf340f9c346800739f2f" exitCode=0 Sep 06 19:42:03.131878 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.131733 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" event={"ID":"d4030ff8-ddae-498b-a194-fbad92585aa8","Type":"ContainerDied","Data":"66f435d3fc1120e258bb8712723c54309ad3fd94f299bf340f9c346800739f2f"} Sep 06 19:42:03.264857 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.264834 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:42:03.267818 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.267791 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:42:03.414935 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.414902 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-plugins-config-volume\") pod \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " Sep 06 19:42:03.415132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.414977 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dvvx\" (UniqueName: \"kubernetes.io/projected/d4030ff8-ddae-498b-a194-fbad92585aa8-kube-api-access-5dvvx\") pod \"d4030ff8-ddae-498b-a194-fbad92585aa8\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " Sep 06 19:42:03.415132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.415012 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4030ff8-ddae-498b-a194-fbad92585aa8-plugins-config-volume\") pod \"d4030ff8-ddae-498b-a194-fbad92585aa8\" (UID: \"d4030ff8-ddae-498b-a194-fbad92585aa8\") " Sep 06 19:42:03.415132 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.415027 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhtkb\" (UniqueName: \"kubernetes.io/projected/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-kube-api-access-jhtkb\") pod \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\" (UID: \"1b7c2d49-d326-4238-be96-ca89f1a3fd4a\") " Sep 06 19:42:03.415380 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.415350 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4030ff8-ddae-498b-a194-fbad92585aa8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4030ff8-ddae-498b-a194-fbad92585aa8" (UID: "d4030ff8-ddae-498b-a194-fbad92585aa8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:42:03.415434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.415350 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b7c2d49-d326-4238-be96-ca89f1a3fd4a" (UID: "1b7c2d49-d326-4238-be96-ca89f1a3fd4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:42:03.417376 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.417350 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-kube-api-access-jhtkb" (OuterVolumeSpecName: "kube-api-access-jhtkb") pod "1b7c2d49-d326-4238-be96-ca89f1a3fd4a" (UID: "1b7c2d49-d326-4238-be96-ca89f1a3fd4a"). InnerVolumeSpecName "kube-api-access-jhtkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:42:03.417472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.417404 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4030ff8-ddae-498b-a194-fbad92585aa8-kube-api-access-5dvvx" (OuterVolumeSpecName: "kube-api-access-5dvvx") pod "d4030ff8-ddae-498b-a194-fbad92585aa8" (UID: "d4030ff8-ddae-498b-a194-fbad92585aa8"). InnerVolumeSpecName "kube-api-access-5dvvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:42:03.516101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.516009 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:03.516101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.516043 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dvvx\" (UniqueName: \"kubernetes.io/projected/d4030ff8-ddae-498b-a194-fbad92585aa8-kube-api-access-5dvvx\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:03.516101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.516055 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4030ff8-ddae-498b-a194-fbad92585aa8-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:03.516101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:03.516067 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhtkb\" (UniqueName: \"kubernetes.io/projected/1b7c2d49-d326-4238-be96-ca89f1a3fd4a-kube-api-access-jhtkb\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:04.140666 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.140584 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" Sep 06 19:42:04.140666 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.140628 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9" event={"ID":"1b7c2d49-d326-4238-be96-ca89f1a3fd4a","Type":"ContainerDied","Data":"ce8fb23b8e43f62daddfc1c1596ac5c55c38dd6fc0260bf44bb0da581ad994c8"} Sep 06 19:42:04.141197 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.140704 2575 scope.go:117] "RemoveContainer" containerID="851eff8e1e646288ef00823887e7c94b2bf54214d11e94649b74cdbc45823d08" Sep 06 19:42:04.142293 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.142266 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" event={"ID":"d4030ff8-ddae-498b-a194-fbad92585aa8","Type":"ContainerDied","Data":"70380fad45436783c68e0e32031eb82aac1fad377da1c330d6cdbca558099e03"} Sep 06 19:42:04.142567 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.142334 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2" Sep 06 19:42:04.151679 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.151659 2575 scope.go:117] "RemoveContainer" containerID="66f435d3fc1120e258bb8712723c54309ad3fd94f299bf340f9c346800739f2f" Sep 06 19:42:04.166250 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.166224 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2"] Sep 06 19:42:04.171615 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.171590 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4e3f8e3b-6956bdf899-7zbq2"] Sep 06 19:42:04.183916 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.183891 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9"] Sep 06 19:42:04.194003 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:04.193966 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e3f8e3b-6b55d4c45d-qjmn9"] Sep 06 19:42:05.762886 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:05.762856 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b7c2d49-d326-4238-be96-ca89f1a3fd4a" path="/var/lib/kubelet/pods/1b7c2d49-d326-4238-be96-ca89f1a3fd4a/volumes" Sep 06 19:42:05.763227 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:05.763214 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4030ff8-ddae-498b-a194-fbad92585aa8" path="/var/lib/kubelet/pods/d4030ff8-ddae-498b-a194-fbad92585aa8/volumes" Sep 06 19:42:06.092849 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:06.092743 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:42:06.092849 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:06.092843 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:42:07.344208 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344121 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq"] Sep 06 19:42:07.344629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344566 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4030ff8-ddae-498b-a194-fbad92585aa8" containerName="payload-processing" Sep 06 19:42:07.344629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344579 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4030ff8-ddae-498b-a194-fbad92585aa8" containerName="payload-processing" Sep 06 19:42:07.344629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344602 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b7c2d49-d326-4238-be96-ca89f1a3fd4a" containerName="payload-pre-processing" Sep 06 19:42:07.344629 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344608 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7c2d49-d326-4238-be96-ca89f1a3fd4a" containerName="payload-pre-processing" Sep 06 19:42:07.344775 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344664 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b7c2d49-d326-4238-be96-ca89f1a3fd4a" containerName="payload-pre-processing" Sep 06 19:42:07.344775 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.344671 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4030ff8-ddae-498b-a194-fbad92585aa8" containerName="payload-processing" Sep 06 19:42:07.349014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.348997 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.351574 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.351550 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b4020e88-dockercfg-hzn69\"" Sep 06 19:42:07.351698 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.351577 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b4020e88\"" Sep 06 19:42:07.358240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.358216 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq"] Sep 06 19:42:07.387950 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.387918 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx"] Sep 06 19:42:07.391366 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.391351 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.400323 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.400291 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx"] Sep 06 19:42:07.455277 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.455238 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t4w6\" (UniqueName: \"kubernetes.io/projected/bb26c10d-9449-4438-8a1e-534fc9051d0a-kube-api-access-9t4w6\") pod \"payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.455472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.455453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb26c10d-9449-4438-8a1e-534fc9051d0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.556283 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.556249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb26c10d-9449-4438-8a1e-534fc9051d0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.556490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.556296 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t4w6\" (UniqueName: \"kubernetes.io/projected/bb26c10d-9449-4438-8a1e-534fc9051d0a-kube-api-access-9t4w6\") pod \"payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.556490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.556335 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgnt\" (UniqueName: \"kubernetes.io/projected/1d527701-48c3-40f6-8bcc-208f39d36847-kube-api-access-vcgnt\") pod \"payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.556490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.556430 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d527701-48c3-40f6-8bcc-208f39d36847-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.557016 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.556997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb26c10d-9449-4438-8a1e-534fc9051d0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.568072 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.568048 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t4w6\" (UniqueName: \"kubernetes.io/projected/bb26c10d-9449-4438-8a1e-534fc9051d0a-kube-api-access-9t4w6\") pod \"payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.657199 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.657110 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d527701-48c3-40f6-8bcc-208f39d36847-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.657347 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.657209 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgnt\" (UniqueName: \"kubernetes.io/projected/1d527701-48c3-40f6-8bcc-208f39d36847-kube-api-access-vcgnt\") pod \"payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.657838 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.657816 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d527701-48c3-40f6-8bcc-208f39d36847-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.661028 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.661009 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:07.665037 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.665013 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgnt\" (UniqueName: \"kubernetes.io/projected/1d527701-48c3-40f6-8bcc-208f39d36847-kube-api-access-vcgnt\") pod \"payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.702901 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.702868 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:07.938800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.938779 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq"] Sep 06 19:42:07.939679 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:42:07.939658 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb26c10d_9449_4438_8a1e_534fc9051d0a.slice/crio-1ba256a61f35017e11169135965eaa8312259fc463f8037efa3a92f7faaa1f99 WatchSource:0}: Error finding container 1ba256a61f35017e11169135965eaa8312259fc463f8037efa3a92f7faaa1f99: Status 404 returned error can't find the container with id 1ba256a61f35017e11169135965eaa8312259fc463f8037efa3a92f7faaa1f99 Sep 06 19:42:07.998792 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:07.998768 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx"] Sep 06 19:42:08.000729 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:42:08.000703 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d527701_48c3_40f6_8bcc_208f39d36847.slice/crio-ebbbc753f2f46158f54513eddf1f57f47f6b233e9b672643b46ed7679ee985d5 WatchSource:0}: Error finding container ebbbc753f2f46158f54513eddf1f57f47f6b233e9b672643b46ed7679ee985d5: Status 404 returned error can't find the container with id ebbbc753f2f46158f54513eddf1f57f47f6b233e9b672643b46ed7679ee985d5 Sep 06 19:42:08.171990 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.171934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" event={"ID":"1d527701-48c3-40f6-8bcc-208f39d36847","Type":"ContainerStarted","Data":"65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374"} Sep 06 19:42:08.171990 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.171998 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" event={"ID":"1d527701-48c3-40f6-8bcc-208f39d36847","Type":"ContainerStarted","Data":"ebbbc753f2f46158f54513eddf1f57f47f6b233e9b672643b46ed7679ee985d5"} Sep 06 19:42:08.172259 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.172056 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:08.173692 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.173668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" event={"ID":"bb26c10d-9449-4438-8a1e-534fc9051d0a","Type":"ContainerStarted","Data":"d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba"} Sep 06 19:42:08.173842 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.173716 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" event={"ID":"bb26c10d-9449-4438-8a1e-534fc9051d0a","Type":"ContainerStarted","Data":"1ba256a61f35017e11169135965eaa8312259fc463f8037efa3a92f7faaa1f99"} Sep 06 19:42:08.173842 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.173743 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:08.188318 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.188227 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" podStartSLOduration=1.188212551 podStartE2EDuration="1.188212551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:42:08.187926035 +0000 UTC m=+870.922530768" watchObservedRunningTime="2026-09-06 19:42:08.188212551 +0000 UTC m=+870.922817281" Sep 06 19:42:08.208192 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:08.208143 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" podStartSLOduration=1.208130186 podStartE2EDuration="1.208130186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:42:08.207118059 +0000 UTC m=+870.941722790" watchObservedRunningTime="2026-09-06 19:42:08.208130186 +0000 UTC m=+870.942734963" Sep 06 19:42:19.181938 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:19.181909 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:19.182417 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:19.181984 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:37.714982 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:37.714900 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:42:37.715407 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:37.715211 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:42:37.948620 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:37.948592 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx"] Sep 06 19:42:37.948855 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:37.948810 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" podUID="1d527701-48c3-40f6-8bcc-208f39d36847" containerName="payload-processing" containerID="cri-o://65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374" gracePeriod=30 Sep 06 19:42:37.956799 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:37.956773 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq"] Sep 06 19:42:37.957142 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:37.957110 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" podUID="bb26c10d-9449-4438-8a1e-534fc9051d0a" containerName="payload-pre-processing" containerID="cri-o://d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba" gracePeriod=30 Sep 06 19:42:38.247511 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.247479 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:38.250867 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.250844 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:38.350606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.350569 2575 generic.go:356] "Generic (PLEG): container finished" podID="1d527701-48c3-40f6-8bcc-208f39d36847" containerID="65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374" exitCode=0 Sep 06 19:42:38.350606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.350600 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" event={"ID":"1d527701-48c3-40f6-8bcc-208f39d36847","Type":"ContainerDied","Data":"65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374"} Sep 06 19:42:38.350899 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.350622 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" Sep 06 19:42:38.350899 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.350690 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx" event={"ID":"1d527701-48c3-40f6-8bcc-208f39d36847","Type":"ContainerDied","Data":"ebbbc753f2f46158f54513eddf1f57f47f6b233e9b672643b46ed7679ee985d5"} Sep 06 19:42:38.350899 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.350719 2575 scope.go:117] "RemoveContainer" containerID="65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374" Sep 06 19:42:38.351411 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.351387 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t4w6\" (UniqueName: \"kubernetes.io/projected/bb26c10d-9449-4438-8a1e-534fc9051d0a-kube-api-access-9t4w6\") pod \"bb26c10d-9449-4438-8a1e-534fc9051d0a\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " Sep 06 19:42:38.351539 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.351429 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d527701-48c3-40f6-8bcc-208f39d36847-plugins-config-volume\") pod \"1d527701-48c3-40f6-8bcc-208f39d36847\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " Sep 06 19:42:38.351606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.351555 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb26c10d-9449-4438-8a1e-534fc9051d0a-plugins-config-volume\") pod \"bb26c10d-9449-4438-8a1e-534fc9051d0a\" (UID: \"bb26c10d-9449-4438-8a1e-534fc9051d0a\") " Sep 06 19:42:38.351732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.351659 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcgnt\" (UniqueName: \"kubernetes.io/projected/1d527701-48c3-40f6-8bcc-208f39d36847-kube-api-access-vcgnt\") pod \"1d527701-48c3-40f6-8bcc-208f39d36847\" (UID: \"1d527701-48c3-40f6-8bcc-208f39d36847\") " Sep 06 19:42:38.351930 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.351908 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d527701-48c3-40f6-8bcc-208f39d36847-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d527701-48c3-40f6-8bcc-208f39d36847" (UID: "1d527701-48c3-40f6-8bcc-208f39d36847"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:42:38.351998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.351954 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb26c10d-9449-4438-8a1e-534fc9051d0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb26c10d-9449-4438-8a1e-534fc9051d0a" (UID: "bb26c10d-9449-4438-8a1e-534fc9051d0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:42:38.352120 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.352102 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d527701-48c3-40f6-8bcc-208f39d36847-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:38.352168 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.352126 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb26c10d-9449-4438-8a1e-534fc9051d0a-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:38.352227 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.352207 2575 generic.go:356] "Generic (PLEG): container finished" podID="bb26c10d-9449-4438-8a1e-534fc9051d0a" containerID="d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba" exitCode=0 Sep 06 19:42:38.352391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.352311 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" event={"ID":"bb26c10d-9449-4438-8a1e-534fc9051d0a","Type":"ContainerDied","Data":"d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba"} Sep 06 19:42:38.352391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.352362 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" event={"ID":"bb26c10d-9449-4438-8a1e-534fc9051d0a","Type":"ContainerDied","Data":"1ba256a61f35017e11169135965eaa8312259fc463f8037efa3a92f7faaa1f99"} Sep 06 19:42:38.352490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.352428 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq" Sep 06 19:42:38.353923 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.353894 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d527701-48c3-40f6-8bcc-208f39d36847-kube-api-access-vcgnt" (OuterVolumeSpecName: "kube-api-access-vcgnt") pod "1d527701-48c3-40f6-8bcc-208f39d36847" (UID: "1d527701-48c3-40f6-8bcc-208f39d36847"). InnerVolumeSpecName "kube-api-access-vcgnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:42:38.354054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.354040 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb26c10d-9449-4438-8a1e-534fc9051d0a-kube-api-access-9t4w6" (OuterVolumeSpecName: "kube-api-access-9t4w6") pod "bb26c10d-9449-4438-8a1e-534fc9051d0a" (UID: "bb26c10d-9449-4438-8a1e-534fc9051d0a"). InnerVolumeSpecName "kube-api-access-9t4w6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:42:38.360059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.360041 2575 scope.go:117] "RemoveContainer" containerID="65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374" Sep 06 19:42:38.360319 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:42:38.360297 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374\": container with ID starting with 65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374 not found: ID does not exist" containerID="65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374" Sep 06 19:42:38.360394 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.360330 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374"} err="failed to get container status \"65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374\": rpc error: code = NotFound desc = could not find container \"65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374\": container with ID starting with 65e2527dda45d1a08641e910814f26f7d0ca0d1236b39f38d441816871bec374 not found: ID does not exist" Sep 06 19:42:38.360394 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.360351 2575 scope.go:117] "RemoveContainer" containerID="d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba" Sep 06 19:42:38.368384 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.368369 2575 scope.go:117] "RemoveContainer" containerID="d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba" Sep 06 19:42:38.368652 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:42:38.368632 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba\": container with ID starting with d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba not found: ID does not exist" containerID="d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba" Sep 06 19:42:38.368702 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.368659 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba"} err="failed to get container status \"d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba\": rpc error: code = NotFound desc = could not find container \"d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba\": container with ID starting with d50402d73370c211a64a9d0967801320d513baf83e35759bc9ae30f65bd1f2ba not found: ID does not exist" Sep 06 19:42:38.453311 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.453262 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t4w6\" (UniqueName: \"kubernetes.io/projected/bb26c10d-9449-4438-8a1e-534fc9051d0a-kube-api-access-9t4w6\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:38.453311 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.453306 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcgnt\" (UniqueName: \"kubernetes.io/projected/1d527701-48c3-40f6-8bcc-208f39d36847-kube-api-access-vcgnt\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:42:38.672936 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.672913 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx"] Sep 06 19:42:38.678848 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.678827 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b4020e88-68f4b64bc8-q55wx"] Sep 06 19:42:38.687940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.687917 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq"] Sep 06 19:42:38.691601 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:38.691579 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b4020e88-577bc67698-8pmdq"] Sep 06 19:42:39.762306 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:39.762271 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d527701-48c3-40f6-8bcc-208f39d36847" path="/var/lib/kubelet/pods/1d527701-48c3-40f6-8bcc-208f39d36847/volumes" Sep 06 19:42:39.762676 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:39.762618 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb26c10d-9449-4438-8a1e-534fc9051d0a" path="/var/lib/kubelet/pods/bb26c10d-9449-4438-8a1e-534fc9051d0a/volumes" Sep 06 19:42:43.372729 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.372694 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt"] Sep 06 19:42:43.373119 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.373103 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d527701-48c3-40f6-8bcc-208f39d36847" containerName="payload-processing" Sep 06 19:42:43.373119 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.373120 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d527701-48c3-40f6-8bcc-208f39d36847" containerName="payload-processing" Sep 06 19:42:43.373191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.373130 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb26c10d-9449-4438-8a1e-534fc9051d0a" containerName="payload-pre-processing" Sep 06 19:42:43.373191 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.373136 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb26c10d-9449-4438-8a1e-534fc9051d0a" containerName="payload-pre-processing" Sep 06 19:42:43.373260 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.373203 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d527701-48c3-40f6-8bcc-208f39d36847" containerName="payload-processing" Sep 06 19:42:43.373260 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.373213 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb26c10d-9449-4438-8a1e-534fc9051d0a" containerName="payload-pre-processing" Sep 06 19:42:43.377892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.377873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.380633 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.380609 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b6954964\"" Sep 06 19:42:43.386825 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.386805 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt"] Sep 06 19:42:43.414336 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.414302 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj"] Sep 06 19:42:43.417687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.417667 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.420608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.420590 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b6954964-dockercfg-8wl64\"" Sep 06 19:42:43.427959 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.427937 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj"] Sep 06 19:42:43.499630 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.499595 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063befeb-2bb8-404b-b677-fc2728f9567a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.499931 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.499648 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade3e75-2b6a-44f4-b180-aeb71e529854-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.499931 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.499807 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh84n\" (UniqueName: \"kubernetes.io/projected/eade3e75-2b6a-44f4-b180-aeb71e529854-kube-api-access-nh84n\") pod \"payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.500063 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.499957 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-527bm\" (UniqueName: \"kubernetes.io/projected/063befeb-2bb8-404b-b677-fc2728f9567a-kube-api-access-527bm\") pod \"payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.601222 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.601190 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-527bm\" (UniqueName: \"kubernetes.io/projected/063befeb-2bb8-404b-b677-fc2728f9567a-kube-api-access-527bm\") pod \"payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.601391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.601268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063befeb-2bb8-404b-b677-fc2728f9567a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.601391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.601286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade3e75-2b6a-44f4-b180-aeb71e529854-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.601391 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.601331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh84n\" (UniqueName: \"kubernetes.io/projected/eade3e75-2b6a-44f4-b180-aeb71e529854-kube-api-access-nh84n\") pod \"payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.602016 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.601996 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063befeb-2bb8-404b-b677-fc2728f9567a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.602063 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.602013 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade3e75-2b6a-44f4-b180-aeb71e529854-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.608628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.608608 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-527bm\" (UniqueName: \"kubernetes.io/projected/063befeb-2bb8-404b-b677-fc2728f9567a-kube-api-access-527bm\") pod \"payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.608734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.608693 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh84n\" (UniqueName: \"kubernetes.io/projected/eade3e75-2b6a-44f4-b180-aeb71e529854-kube-api-access-nh84n\") pod \"payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.689333 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.689309 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:43.728866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.728831 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:43.966602 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:43.966551 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt"] Sep 06 19:42:43.969330 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:42:43.969298 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeade3e75_2b6a_44f4_b180_aeb71e529854.slice/crio-5242d33c6ac579985b3cbbb1d2771929c90c7a0f9447fbb5d481d83a0e4dbf30 WatchSource:0}: Error finding container 5242d33c6ac579985b3cbbb1d2771929c90c7a0f9447fbb5d481d83a0e4dbf30: Status 404 returned error can't find the container with id 5242d33c6ac579985b3cbbb1d2771929c90c7a0f9447fbb5d481d83a0e4dbf30 Sep 06 19:42:44.012417 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.012392 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj"] Sep 06 19:42:44.014802 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:42:44.014778 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063befeb_2bb8_404b_b677_fc2728f9567a.slice/crio-db96bb0070e3eebbc76c10aa27a1e861582bc82c0e52b330b97cafa6a469ee4d WatchSource:0}: Error finding container db96bb0070e3eebbc76c10aa27a1e861582bc82c0e52b330b97cafa6a469ee4d: Status 404 returned error can't find the container with id db96bb0070e3eebbc76c10aa27a1e861582bc82c0e52b330b97cafa6a469ee4d Sep 06 19:42:44.397916 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.397815 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" event={"ID":"063befeb-2bb8-404b-b677-fc2728f9567a","Type":"ContainerStarted","Data":"5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55"} Sep 06 19:42:44.397916 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.397877 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" event={"ID":"063befeb-2bb8-404b-b677-fc2728f9567a","Type":"ContainerStarted","Data":"db96bb0070e3eebbc76c10aa27a1e861582bc82c0e52b330b97cafa6a469ee4d"} Sep 06 19:42:44.398401 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.398024 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:42:44.399570 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.399542 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" event={"ID":"eade3e75-2b6a-44f4-b180-aeb71e529854","Type":"ContainerStarted","Data":"3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb"} Sep 06 19:42:44.399707 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.399588 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" event={"ID":"eade3e75-2b6a-44f4-b180-aeb71e529854","Type":"ContainerStarted","Data":"5242d33c6ac579985b3cbbb1d2771929c90c7a0f9447fbb5d481d83a0e4dbf30"} Sep 06 19:42:44.399707 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.399630 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:44.414797 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.414731 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" podStartSLOduration=1.4147146290000001 podStartE2EDuration="1.414714629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:42:44.411409309 +0000 UTC m=+907.146014064" watchObservedRunningTime="2026-09-06 19:42:44.414714629 +0000 UTC m=+907.149319362" Sep 06 19:42:44.429158 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:44.429105 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" podStartSLOduration=1.429089101 podStartE2EDuration="1.429089101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:42:44.425391754 +0000 UTC m=+907.159996486" watchObservedRunningTime="2026-09-06 19:42:44.429089101 +0000 UTC m=+907.163693831" Sep 06 19:42:55.410298 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:55.410265 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:42:55.410943 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:42:55.410443 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:43:13.629383 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.629349 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj"] Sep 06 19:43:13.629852 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.629659 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" podUID="063befeb-2bb8-404b-b677-fc2728f9567a" containerName="payload-processing" containerID="cri-o://5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55" gracePeriod=30 Sep 06 19:43:13.636458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.636428 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt"] Sep 06 19:43:13.636886 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.636835 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" podUID="eade3e75-2b6a-44f4-b180-aeb71e529854" containerName="payload-pre-processing" containerID="cri-o://3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb" gracePeriod=30 Sep 06 19:43:13.915886 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.915863 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:43:13.918731 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.918711 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:43:13.990048 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990019 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade3e75-2b6a-44f4-b180-aeb71e529854-plugins-config-volume\") pod \"eade3e75-2b6a-44f4-b180-aeb71e529854\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " Sep 06 19:43:13.990238 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990082 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh84n\" (UniqueName: \"kubernetes.io/projected/eade3e75-2b6a-44f4-b180-aeb71e529854-kube-api-access-nh84n\") pod \"eade3e75-2b6a-44f4-b180-aeb71e529854\" (UID: \"eade3e75-2b6a-44f4-b180-aeb71e529854\") " Sep 06 19:43:13.990238 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990102 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-527bm\" (UniqueName: \"kubernetes.io/projected/063befeb-2bb8-404b-b677-fc2728f9567a-kube-api-access-527bm\") pod \"063befeb-2bb8-404b-b677-fc2728f9567a\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " Sep 06 19:43:13.990238 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990218 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063befeb-2bb8-404b-b677-fc2728f9567a-plugins-config-volume\") pod \"063befeb-2bb8-404b-b677-fc2728f9567a\" (UID: \"063befeb-2bb8-404b-b677-fc2728f9567a\") " Sep 06 19:43:13.990479 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990455 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eade3e75-2b6a-44f4-b180-aeb71e529854-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eade3e75-2b6a-44f4-b180-aeb71e529854" (UID: "eade3e75-2b6a-44f4-b180-aeb71e529854"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:43:13.990547 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990505 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063befeb-2bb8-404b-b677-fc2728f9567a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "063befeb-2bb8-404b-b677-fc2728f9567a" (UID: "063befeb-2bb8-404b-b677-fc2728f9567a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:43:13.990591 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990582 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade3e75-2b6a-44f4-b180-aeb71e529854-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:13.990628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.990596 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063befeb-2bb8-404b-b677-fc2728f9567a-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:13.992237 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.992215 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eade3e75-2b6a-44f4-b180-aeb71e529854-kube-api-access-nh84n" (OuterVolumeSpecName: "kube-api-access-nh84n") pod "eade3e75-2b6a-44f4-b180-aeb71e529854" (UID: "eade3e75-2b6a-44f4-b180-aeb71e529854"). InnerVolumeSpecName "kube-api-access-nh84n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:13.992377 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:13.992357 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063befeb-2bb8-404b-b677-fc2728f9567a-kube-api-access-527bm" (OuterVolumeSpecName: "kube-api-access-527bm") pod "063befeb-2bb8-404b-b677-fc2728f9567a" (UID: "063befeb-2bb8-404b-b677-fc2728f9567a"). InnerVolumeSpecName "kube-api-access-527bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:14.091660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.091629 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh84n\" (UniqueName: \"kubernetes.io/projected/eade3e75-2b6a-44f4-b180-aeb71e529854-kube-api-access-nh84n\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:14.091660 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.091656 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-527bm\" (UniqueName: \"kubernetes.io/projected/063befeb-2bb8-404b-b677-fc2728f9567a-kube-api-access-527bm\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:14.571816 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.571783 2575 generic.go:356] "Generic (PLEG): container finished" podID="eade3e75-2b6a-44f4-b180-aeb71e529854" containerID="3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb" exitCode=0 Sep 06 19:43:14.572014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.571827 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" Sep 06 19:43:14.572014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.571876 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" event={"ID":"eade3e75-2b6a-44f4-b180-aeb71e529854","Type":"ContainerDied","Data":"3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb"} Sep 06 19:43:14.572014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.571931 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt" event={"ID":"eade3e75-2b6a-44f4-b180-aeb71e529854","Type":"ContainerDied","Data":"5242d33c6ac579985b3cbbb1d2771929c90c7a0f9447fbb5d481d83a0e4dbf30"} Sep 06 19:43:14.572014 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.571958 2575 scope.go:117] "RemoveContainer" containerID="3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb" Sep 06 19:43:14.573834 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.573676 2575 generic.go:356] "Generic (PLEG): container finished" podID="063befeb-2bb8-404b-b677-fc2728f9567a" containerID="5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55" exitCode=0 Sep 06 19:43:14.573834 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.573781 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" event={"ID":"063befeb-2bb8-404b-b677-fc2728f9567a","Type":"ContainerDied","Data":"5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55"} Sep 06 19:43:14.573834 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.573822 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" Sep 06 19:43:14.574074 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.573824 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj" event={"ID":"063befeb-2bb8-404b-b677-fc2728f9567a","Type":"ContainerDied","Data":"db96bb0070e3eebbc76c10aa27a1e861582bc82c0e52b330b97cafa6a469ee4d"} Sep 06 19:43:14.581884 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.581843 2575 scope.go:117] "RemoveContainer" containerID="3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb" Sep 06 19:43:14.582184 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:43:14.582154 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb\": container with ID starting with 3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb not found: ID does not exist" containerID="3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb" Sep 06 19:43:14.582287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.582194 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb"} err="failed to get container status \"3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb\": rpc error: code = NotFound desc = could not find container \"3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb\": container with ID starting with 3e9ec10b6e6edfdf42e527107bfce1d97662bb4f96a9e8ae7ccd7f4c634cc0fb not found: ID does not exist" Sep 06 19:43:14.582287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.582212 2575 scope.go:117] "RemoveContainer" containerID="5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55" Sep 06 19:43:14.593404 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.593384 2575 scope.go:117] "RemoveContainer" containerID="5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55" Sep 06 19:43:14.593714 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:43:14.593690 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55\": container with ID starting with 5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55 not found: ID does not exist" containerID="5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55" Sep 06 19:43:14.593783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.593722 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55"} err="failed to get container status \"5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55\": rpc error: code = NotFound desc = could not find container \"5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55\": container with ID starting with 5f9449493587e3d20102962fa72413d21f7dead0be3fdab992b7450e00c95c55 not found: ID does not exist" Sep 06 19:43:14.600319 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.600297 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj"] Sep 06 19:43:14.602707 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.602688 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b6954964-78877dfddf-mcqdj"] Sep 06 19:43:14.614185 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.614160 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt"] Sep 06 19:43:14.619341 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:14.619318 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b6954964-5b7dbbfdd6-6mktt"] Sep 06 19:43:15.564475 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564446 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687"] Sep 06 19:43:15.564975 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564871 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eade3e75-2b6a-44f4-b180-aeb71e529854" containerName="payload-pre-processing" Sep 06 19:43:15.564975 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564885 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eade3e75-2b6a-44f4-b180-aeb71e529854" containerName="payload-pre-processing" Sep 06 19:43:15.564975 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564897 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="063befeb-2bb8-404b-b677-fc2728f9567a" containerName="payload-processing" Sep 06 19:43:15.564975 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564903 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="063befeb-2bb8-404b-b677-fc2728f9567a" containerName="payload-processing" Sep 06 19:43:15.565198 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564980 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="063befeb-2bb8-404b-b677-fc2728f9567a" containerName="payload-processing" Sep 06 19:43:15.565198 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.564992 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="eade3e75-2b6a-44f4-b180-aeb71e529854" containerName="payload-pre-processing" Sep 06 19:43:15.569451 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.569430 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.571769 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.571730 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f439b863-openshift-default-dockercfg-22zhd\"" Sep 06 19:43:15.578302 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.578280 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687"] Sep 06 19:43:15.705486 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705447 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istiod-ca-cert\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705511 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-envoy\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-podinfo\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-certs\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705645 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-data\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705687 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705666 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-token\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705963 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-credential-socket\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705963 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705814 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-socket\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.705963 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.705841 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-kube-api-access-8pn6f\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.764235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.764201 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="063befeb-2bb8-404b-b677-fc2728f9567a" path="/var/lib/kubelet/pods/063befeb-2bb8-404b-b677-fc2728f9567a/volumes" Sep 06 19:43:15.764567 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.764554 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eade3e75-2b6a-44f4-b180-aeb71e529854" path="/var/lib/kubelet/pods/eade3e75-2b6a-44f4-b180-aeb71e529854/volumes" Sep 06 19:43:15.807172 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807127 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istiod-ca-cert\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-envoy\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807213 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-podinfo\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-certs\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-data\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-token\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-credential-socket\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-socket\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-kube-api-access-8pn6f\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807811 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807670 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-certs\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807881 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807851 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-socket\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807944 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807882 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-credential-socket\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.807944 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.807908 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-data\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.808193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.808132 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istiod-ca-cert\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.809789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.809767 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-envoy\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.810100 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.810077 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-podinfo\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.815018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.814954 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-token\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.815098 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.815031 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-kube-api-access-8pn6f\") pod \"e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:15.881584 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:15.881548 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:16.174872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.174847 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687"] Sep 06 19:43:16.176644 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:43:16.176615 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ad1dea_1042_4754_b5f9_6adcfe3840c3.slice/crio-196add8af901a71750df904e60d2ce15ff90c2780bfae56f618468f580b9a811 WatchSource:0}: Error finding container 196add8af901a71750df904e60d2ce15ff90c2780bfae56f618468f580b9a811: Status 404 returned error can't find the container with id 196add8af901a71750df904e60d2ce15ff90c2780bfae56f618468f580b9a811 Sep 06 19:43:16.178795 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.178732 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:43:16.178896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.178832 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:43:16.178896 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.178864 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:43:16.591561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.591464 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" event={"ID":"97ad1dea-1042-4754-b5f9-6adcfe3840c3","Type":"ContainerStarted","Data":"77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa"} Sep 06 19:43:16.591561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.591522 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" event={"ID":"97ad1dea-1042-4754-b5f9-6adcfe3840c3","Type":"ContainerStarted","Data":"196add8af901a71750df904e60d2ce15ff90c2780bfae56f618468f580b9a811"} Sep 06 19:43:16.613997 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.613942 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podStartSLOduration=1.613921914 podStartE2EDuration="1.613921914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:43:16.610292022 +0000 UTC m=+939.344896753" watchObservedRunningTime="2026-09-06 19:43:16.613921914 +0000 UTC m=+939.348526646" Sep 06 19:43:16.881913 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:16.881826 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:17.886825 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:17.886797 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:18.603877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:18.603850 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:18.604790 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:18.604772 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:19.863817 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.863774 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5"] Sep 06 19:43:19.867807 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.867787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:19.870523 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.870504 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-f439b863-dockercfg-pxls7\"" Sep 06 19:43:19.870651 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.870526 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-f439b863\"" Sep 06 19:43:19.876679 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.876657 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5"] Sep 06 19:43:19.899322 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.899295 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v"] Sep 06 19:43:19.902860 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.902839 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:19.915312 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.915287 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v"] Sep 06 19:43:19.952373 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.952337 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b4f024-3d94-4e01-875c-4611f6d9c918-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:19.952535 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:19.952451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2srg\" (UniqueName: \"kubernetes.io/projected/f2b4f024-3d94-4e01-875c-4611f6d9c918-kube-api-access-v2srg\") pod \"payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.053531 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.053493 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b4f024-3d94-4e01-875c-4611f6d9c918-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.053709 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.053583 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-plugins-config-volume\") pod \"payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.053709 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.053614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2srg\" (UniqueName: \"kubernetes.io/projected/f2b4f024-3d94-4e01-875c-4611f6d9c918-kube-api-access-v2srg\") pod \"payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.053709 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.053642 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxhjj\" (UniqueName: \"kubernetes.io/projected/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-kube-api-access-mxhjj\") pod \"payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.054368 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.054340 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b4f024-3d94-4e01-875c-4611f6d9c918-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.061220 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.061201 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2srg\" (UniqueName: \"kubernetes.io/projected/f2b4f024-3d94-4e01-875c-4611f6d9c918-kube-api-access-v2srg\") pod \"payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.154711 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.154629 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-plugins-config-volume\") pod \"payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.154711 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.154672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxhjj\" (UniqueName: \"kubernetes.io/projected/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-kube-api-access-mxhjj\") pod \"payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.155296 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.155275 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-plugins-config-volume\") pod \"payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.163985 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.163961 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxhjj\" (UniqueName: \"kubernetes.io/projected/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-kube-api-access-mxhjj\") pod \"payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.180432 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.180408 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.213914 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.213886 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.455205 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.455174 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5"] Sep 06 19:43:20.456622 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:43:20.456596 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b4f024_3d94_4e01_875c_4611f6d9c918.slice/crio-24cdead26c4c54421ce6bc8439f765fbde1a4edb55feef21d2bd680152df26fa WatchSource:0}: Error finding container 24cdead26c4c54421ce6bc8439f765fbde1a4edb55feef21d2bd680152df26fa: Status 404 returned error can't find the container with id 24cdead26c4c54421ce6bc8439f765fbde1a4edb55feef21d2bd680152df26fa Sep 06 19:43:20.498101 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.498073 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v"] Sep 06 19:43:20.502795 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:43:20.502773 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3c6210_3afb_414d_9c61_a541c2f2f6f4.slice/crio-a555e0215015b5f925c90da06ed806a19287735ef1fbb792ec8710a36bd4050a WatchSource:0}: Error finding container a555e0215015b5f925c90da06ed806a19287735ef1fbb792ec8710a36bd4050a: Status 404 returned error can't find the container with id a555e0215015b5f925c90da06ed806a19287735ef1fbb792ec8710a36bd4050a Sep 06 19:43:20.624677 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.624640 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" event={"ID":"4b3c6210-3afb-414d-9c61-a541c2f2f6f4","Type":"ContainerStarted","Data":"3b783995fb4de25d4b154920f9ec4fe521e575bada86d43200403a4d71fe9ac0"} Sep 06 19:43:20.624877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.624718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" event={"ID":"4b3c6210-3afb-414d-9c61-a541c2f2f6f4","Type":"ContainerStarted","Data":"a555e0215015b5f925c90da06ed806a19287735ef1fbb792ec8710a36bd4050a"} Sep 06 19:43:20.624877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.624765 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:20.627524 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.626981 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" event={"ID":"f2b4f024-3d94-4e01-875c-4611f6d9c918","Type":"ContainerStarted","Data":"b5709e36d3de21a9d7ef1400bad0cf45a8a404b5fb1aa38788e1caa588197b60"} Sep 06 19:43:20.627832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.627801 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" event={"ID":"f2b4f024-3d94-4e01-875c-4611f6d9c918","Type":"ContainerStarted","Data":"24cdead26c4c54421ce6bc8439f765fbde1a4edb55feef21d2bd680152df26fa"} Sep 06 19:43:20.628059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.628035 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:20.643177 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.643115 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" podStartSLOduration=1.643097295 podStartE2EDuration="1.643097295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:43:20.64051647 +0000 UTC m=+943.375121203" watchObservedRunningTime="2026-09-06 19:43:20.643097295 +0000 UTC m=+943.377702028" Sep 06 19:43:20.656079 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:20.656018 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" podStartSLOduration=1.655998308 podStartE2EDuration="1.655998308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:43:20.654590083 +0000 UTC m=+943.389194816" watchObservedRunningTime="2026-09-06 19:43:20.655998308 +0000 UTC m=+943.390603040" Sep 06 19:43:31.634383 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:31.634350 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:31.634907 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:31.634555 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:41.762759 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:41.762725 2575 generic.go:356] "Generic (PLEG): container finished" podID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerID="77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa" exitCode=0 Sep 06 19:43:41.763163 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:41.762773 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" event={"ID":"97ad1dea-1042-4754-b5f9-6adcfe3840c3","Type":"ContainerDied","Data":"77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa"} Sep 06 19:43:41.763163 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:41.763137 2575 scope.go:117] "RemoveContainer" containerID="77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa" Sep 06 19:43:41.764177 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:41.764151 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:43:41.764244 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:41.764227 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:43:41.764287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:41.764263 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:43:42.770354 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:42.770324 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" event={"ID":"97ad1dea-1042-4754-b5f9-6adcfe3840c3","Type":"ContainerStarted","Data":"af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d"} Sep 06 19:43:42.881693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:42.881662 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:42.881932 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:42.881908 2575 patch_prober.go:28] interesting pod/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 06 19:43:42.881980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:42.881960 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 06 19:43:43.657901 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.657010 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x"] Sep 06 19:43:43.657901 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.657295 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" podUID="efe8e164-e1fb-4ade-b5e4-69cbfbababa0" containerName="payload-pre-processing" containerID="cri-o://809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6" gracePeriod=30 Sep 06 19:43:43.882888 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.882848 2575 patch_prober.go:28] interesting pod/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 06 19:43:43.883355 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.882927 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 06 19:43:43.953503 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.953474 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:43:43.987188 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.987149 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvrm4\" (UniqueName: \"kubernetes.io/projected/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-kube-api-access-qvrm4\") pod \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " Sep 06 19:43:43.987369 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.987273 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-plugins-config-volume\") pod \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\" (UID: \"efe8e164-e1fb-4ade-b5e4-69cbfbababa0\") " Sep 06 19:43:43.987614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.987592 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "efe8e164-e1fb-4ade-b5e4-69cbfbababa0" (UID: "efe8e164-e1fb-4ade-b5e4-69cbfbababa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:43:43.989653 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:43.989616 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-kube-api-access-qvrm4" (OuterVolumeSpecName: "kube-api-access-qvrm4") pod "efe8e164-e1fb-4ade-b5e4-69cbfbababa0" (UID: "efe8e164-e1fb-4ade-b5e4-69cbfbababa0"). InnerVolumeSpecName "kube-api-access-qvrm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:44.088091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.088043 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:44.088091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.088084 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvrm4\" (UniqueName: \"kubernetes.io/projected/efe8e164-e1fb-4ade-b5e4-69cbfbababa0-kube-api-access-qvrm4\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:44.784599 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.784563 2575 generic.go:356] "Generic (PLEG): container finished" podID="efe8e164-e1fb-4ade-b5e4-69cbfbababa0" containerID="809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6" exitCode=0 Sep 06 19:43:44.784832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.784607 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" Sep 06 19:43:44.784832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.784654 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" event={"ID":"efe8e164-e1fb-4ade-b5e4-69cbfbababa0","Type":"ContainerDied","Data":"809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6"} Sep 06 19:43:44.784832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.784715 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x" event={"ID":"efe8e164-e1fb-4ade-b5e4-69cbfbababa0","Type":"ContainerDied","Data":"3fd7256e54883a249bbbf5c6036c16f92989e26e5edf9c7886fdca4c92065fbc"} Sep 06 19:43:44.784832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.784737 2575 scope.go:117] "RemoveContainer" containerID="809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6" Sep 06 19:43:44.795334 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.795310 2575 scope.go:117] "RemoveContainer" containerID="809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6" Sep 06 19:43:44.795689 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:43:44.795663 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6\": container with ID starting with 809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6 not found: ID does not exist" containerID="809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6" Sep 06 19:43:44.795790 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.795701 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6"} err="failed to get container status \"809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6\": rpc error: code = NotFound desc = could not find container \"809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6\": container with ID starting with 809897f11305fd1c3274b14d1db657e1f0e2e42bdf9aadab1236fc2a40e2c5c6 not found: ID does not exist" Sep 06 19:43:44.809824 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.809789 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x"] Sep 06 19:43:44.817966 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.817933 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ebbc9-5ff68ff98-w7r4x"] Sep 06 19:43:44.881936 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.881900 2575 patch_prober.go:28] interesting pod/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 06 19:43:44.882116 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:44.881964 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 06 19:43:45.763262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:45.763225 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe8e164-e1fb-4ade-b5e4-69cbfbababa0" path="/var/lib/kubelet/pods/efe8e164-e1fb-4ade-b5e4-69cbfbababa0/volumes" Sep 06 19:43:45.882631 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:45.882592 2575 patch_prober.go:28] interesting pod/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 06 19:43:45.882873 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:45.882669 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 06 19:43:46.882936 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:46.882900 2575 patch_prober.go:28] interesting pod/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 06 19:43:46.883294 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:46.882957 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 06 19:43:47.886612 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:47.886578 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:47.886995 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:47.886896 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:47.887502 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:47.887486 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:49.661911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.661874 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v"] Sep 06 19:43:49.662376 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.662166 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" podUID="4b3c6210-3afb-414d-9c61-a541c2f2f6f4" containerName="payload-processing" containerID="cri-o://3b783995fb4de25d4b154920f9ec4fe521e575bada86d43200403a4d71fe9ac0" gracePeriod=30 Sep 06 19:43:49.669973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.669944 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5"] Sep 06 19:43:49.670386 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.670174 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" podUID="f2b4f024-3d94-4e01-875c-4611f6d9c918" containerName="payload-pre-processing" containerID="cri-o://b5709e36d3de21a9d7ef1400bad0cf45a8a404b5fb1aa38788e1caa588197b60" gracePeriod=30 Sep 06 19:43:49.822015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.821925 2575 generic.go:356] "Generic (PLEG): container finished" podID="4b3c6210-3afb-414d-9c61-a541c2f2f6f4" containerID="3b783995fb4de25d4b154920f9ec4fe521e575bada86d43200403a4d71fe9ac0" exitCode=0 Sep 06 19:43:49.822015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.821943 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" event={"ID":"4b3c6210-3afb-414d-9c61-a541c2f2f6f4","Type":"ContainerDied","Data":"3b783995fb4de25d4b154920f9ec4fe521e575bada86d43200403a4d71fe9ac0"} Sep 06 19:43:49.823769 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.823728 2575 generic.go:356] "Generic (PLEG): container finished" podID="f2b4f024-3d94-4e01-875c-4611f6d9c918" containerID="b5709e36d3de21a9d7ef1400bad0cf45a8a404b5fb1aa38788e1caa588197b60" exitCode=0 Sep 06 19:43:49.824007 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.823944 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" event={"ID":"f2b4f024-3d94-4e01-875c-4611f6d9c918","Type":"ContainerDied","Data":"b5709e36d3de21a9d7ef1400bad0cf45a8a404b5fb1aa38788e1caa588197b60"} Sep 06 19:43:49.930528 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.930485 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687"] Sep 06 19:43:49.955559 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.955529 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:49.987597 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:49.987571 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:50.059898 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.059865 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2srg\" (UniqueName: \"kubernetes.io/projected/f2b4f024-3d94-4e01-875c-4611f6d9c918-kube-api-access-v2srg\") pod \"f2b4f024-3d94-4e01-875c-4611f6d9c918\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " Sep 06 19:43:50.060105 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.059984 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-plugins-config-volume\") pod \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " Sep 06 19:43:50.060189 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.060166 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxhjj\" (UniqueName: \"kubernetes.io/projected/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-kube-api-access-mxhjj\") pod \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\" (UID: \"4b3c6210-3afb-414d-9c61-a541c2f2f6f4\") " Sep 06 19:43:50.060368 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.060336 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b3c6210-3afb-414d-9c61-a541c2f2f6f4" (UID: "4b3c6210-3afb-414d-9c61-a541c2f2f6f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:43:50.060472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.060422 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b4f024-3d94-4e01-875c-4611f6d9c918-plugins-config-volume\") pod \"f2b4f024-3d94-4e01-875c-4611f6d9c918\" (UID: \"f2b4f024-3d94-4e01-875c-4611f6d9c918\") " Sep 06 19:43:50.060960 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.060936 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2b4f024-3d94-4e01-875c-4611f6d9c918-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2b4f024-3d94-4e01-875c-4611f6d9c918" (UID: "f2b4f024-3d94-4e01-875c-4611f6d9c918"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:43:50.062919 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.062890 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:50.063025 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.062923 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b4f024-3d94-4e01-875c-4611f6d9c918-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:50.063288 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.063265 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b4f024-3d94-4e01-875c-4611f6d9c918-kube-api-access-v2srg" (OuterVolumeSpecName: "kube-api-access-v2srg") pod "f2b4f024-3d94-4e01-875c-4611f6d9c918" (UID: "f2b4f024-3d94-4e01-875c-4611f6d9c918"). InnerVolumeSpecName "kube-api-access-v2srg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:50.063377 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.063355 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-kube-api-access-mxhjj" (OuterVolumeSpecName: "kube-api-access-mxhjj") pod "4b3c6210-3afb-414d-9c61-a541c2f2f6f4" (UID: "4b3c6210-3afb-414d-9c61-a541c2f2f6f4"). InnerVolumeSpecName "kube-api-access-mxhjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:50.164392 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.164362 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2srg\" (UniqueName: \"kubernetes.io/projected/f2b4f024-3d94-4e01-875c-4611f6d9c918-kube-api-access-v2srg\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:50.164392 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.164388 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxhjj\" (UniqueName: \"kubernetes.io/projected/4b3c6210-3afb-414d-9c61-a541c2f2f6f4-kube-api-access-mxhjj\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:50.830462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.830433 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" Sep 06 19:43:50.830462 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.830461 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v" event={"ID":"4b3c6210-3afb-414d-9c61-a541c2f2f6f4","Type":"ContainerDied","Data":"a555e0215015b5f925c90da06ed806a19287735ef1fbb792ec8710a36bd4050a"} Sep 06 19:43:50.830985 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.830524 2575 scope.go:117] "RemoveContainer" containerID="3b783995fb4de25d4b154920f9ec4fe521e575bada86d43200403a4d71fe9ac0" Sep 06 19:43:50.831974 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.831957 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" Sep 06 19:43:50.832058 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.832037 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5" event={"ID":"f2b4f024-3d94-4e01-875c-4611f6d9c918","Type":"ContainerDied","Data":"24cdead26c4c54421ce6bc8439f765fbde1a4edb55feef21d2bd680152df26fa"} Sep 06 19:43:50.832294 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.832236 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" containerID="cri-o://af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d" gracePeriod=30 Sep 06 19:43:50.842208 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.842186 2575 scope.go:117] "RemoveContainer" containerID="b5709e36d3de21a9d7ef1400bad0cf45a8a404b5fb1aa38788e1caa588197b60" Sep 06 19:43:50.870108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.870079 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5"] Sep 06 19:43:50.877848 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.877825 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f439b863-5bb76cd7d6-c8mv5"] Sep 06 19:43:50.889224 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.889194 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v"] Sep 06 19:43:50.892252 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:50.892223 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f439b863-5b59f5dd9-95v5v"] Sep 06 19:43:51.762853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:51.762823 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b3c6210-3afb-414d-9c61-a541c2f2f6f4" path="/var/lib/kubelet/pods/4b3c6210-3afb-414d-9c61-a541c2f2f6f4/volumes" Sep 06 19:43:51.763251 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:51.763237 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b4f024-3d94-4e01-875c-4611f6d9c918" path="/var/lib/kubelet/pods/f2b4f024-3d94-4e01-875c-4611f6d9c918/volumes" Sep 06 19:43:56.088741 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.088712 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:56.221883 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.221848 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-credential-socket\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222158 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222136 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-kube-api-access-8pn6f\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222370 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222341 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-podinfo\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222380 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-token\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222408 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-certs\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222408 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:43:56.222493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222476 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istiod-ca-cert\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222507 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-envoy\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222603 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:43:56.222674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222548 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-socket\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222707 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-data\") pod \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\" (UID: \"97ad1dea-1042-4754-b5f9-6adcfe3840c3\") " Sep 06 19:43:56.222913 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222883 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:43:56.222971 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.222877 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:43:56.223071 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.223028 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-data" (OuterVolumeSpecName: "istio-data") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:43:56.223286 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.223267 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istiod-ca-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.223332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.223295 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.223332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.223309 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-data\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.223332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.223323 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-credential-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.223421 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.223338 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-workload-certs\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.224614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.224588 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-kube-api-access-8pn6f" (OuterVolumeSpecName: "kube-api-access-8pn6f") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "kube-api-access-8pn6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:56.224802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.224772 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:43:56.225046 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.225028 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 19:43:56.225123 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.225103 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-token" (OuterVolumeSpecName: "istio-token") pod "97ad1dea-1042-4754-b5f9-6adcfe3840c3" (UID: "97ad1dea-1042-4754-b5f9-6adcfe3840c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:43:56.324129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.324096 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-podinfo\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.324129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.324123 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-token\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.324129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.324132 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97ad1dea-1042-4754-b5f9-6adcfe3840c3-istio-envoy\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.324362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.324140 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/97ad1dea-1042-4754-b5f9-6adcfe3840c3-kube-api-access-8pn6f\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:43:56.478164 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478082 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk"] Sep 06 19:43:56.478452 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478439 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b4f024-3d94-4e01-875c-4611f6d9c918" containerName="payload-pre-processing" Sep 06 19:43:56.478493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478453 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b4f024-3d94-4e01-875c-4611f6d9c918" containerName="payload-pre-processing" Sep 06 19:43:56.478493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478464 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe8e164-e1fb-4ade-b5e4-69cbfbababa0" containerName="payload-pre-processing" Sep 06 19:43:56.478493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478470 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe8e164-e1fb-4ade-b5e4-69cbfbababa0" containerName="payload-pre-processing" Sep 06 19:43:56.478493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478487 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" Sep 06 19:43:56.478493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478492 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478498 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478503 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478509 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b3c6210-3afb-414d-9c61-a541c2f2f6f4" containerName="payload-processing" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478513 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3c6210-3afb-414d-9c61-a541c2f2f6f4" containerName="payload-processing" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478568 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b3c6210-3afb-414d-9c61-a541c2f2f6f4" containerName="payload-processing" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478578 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b4f024-3d94-4e01-875c-4611f6d9c918" containerName="payload-pre-processing" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478585 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe8e164-e1fb-4ade-b5e4-69cbfbababa0" containerName="payload-pre-processing" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478591 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" Sep 06 19:43:56.478643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.478596 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerName="istio-proxy" Sep 06 19:43:56.480969 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.480955 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.483528 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.483498 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-838def-dockercfg-hvvjb\"" Sep 06 19:43:56.483668 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.483632 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-838def\"" Sep 06 19:43:56.491722 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.491696 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk"] Sep 06 19:43:56.627849 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.627815 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vgqf\" (UniqueName: \"kubernetes.io/projected/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-kube-api-access-6vgqf\") pod \"payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.628048 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.627866 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.728714 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.728623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vgqf\" (UniqueName: \"kubernetes.io/projected/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-kube-api-access-6vgqf\") pod \"payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.728714 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.728667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.729384 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.729365 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.736192 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.736167 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vgqf\" (UniqueName: \"kubernetes.io/projected/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-kube-api-access-6vgqf\") pod \"payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.792273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.792243 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:56.874684 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.874644 2575 generic.go:356] "Generic (PLEG): container finished" podID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" containerID="af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d" exitCode=0 Sep 06 19:43:56.874866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.874719 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" event={"ID":"97ad1dea-1042-4754-b5f9-6adcfe3840c3","Type":"ContainerDied","Data":"af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d"} Sep 06 19:43:56.874866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.874818 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" event={"ID":"97ad1dea-1042-4754-b5f9-6adcfe3840c3","Type":"ContainerDied","Data":"196add8af901a71750df904e60d2ce15ff90c2780bfae56f618468f580b9a811"} Sep 06 19:43:56.874866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.874830 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687" Sep 06 19:43:56.874866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.874853 2575 scope.go:117] "RemoveContainer" containerID="af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d" Sep 06 19:43:56.887730 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.887709 2575 scope.go:117] "RemoveContainer" containerID="77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa" Sep 06 19:43:56.897085 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.897051 2575 scope.go:117] "RemoveContainer" containerID="af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d" Sep 06 19:43:56.897475 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:43:56.897445 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d\": container with ID starting with af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d not found: ID does not exist" containerID="af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d" Sep 06 19:43:56.897567 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.897487 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d"} err="failed to get container status \"af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d\": rpc error: code = NotFound desc = could not find container \"af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d\": container with ID starting with af51cf2d307c5d8161109416de51cbb52c5072163657adf4ac2b51863b5e837d not found: ID does not exist" Sep 06 19:43:56.897567 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.897513 2575 scope.go:117] "RemoveContainer" containerID="77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa" Sep 06 19:43:56.897860 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:43:56.897831 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa\": container with ID starting with 77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa not found: ID does not exist" containerID="77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa" Sep 06 19:43:56.897911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.897867 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa"} err="failed to get container status \"77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa\": rpc error: code = NotFound desc = could not find container \"77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa\": container with ID starting with 77691154d312b19e92b211f6ed520df2681b56c1c590b52f71c0f7b3c74d6dfa not found: ID does not exist" Sep 06 19:43:56.899837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.899817 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687"] Sep 06 19:43:56.905418 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:56.905394 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f439b863-openshift-default-cddcf6cf6-vq687"] Sep 06 19:43:57.061862 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:57.061835 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk"] Sep 06 19:43:57.063653 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:43:57.063629 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c93023_4b6d_46d9_b6a4_5d6c5d3b27e5.slice/crio-45445e8a6e258633a9440c276321099799bcd2d9c142a883380f20a42977f663 WatchSource:0}: Error finding container 45445e8a6e258633a9440c276321099799bcd2d9c142a883380f20a42977f663: Status 404 returned error can't find the container with id 45445e8a6e258633a9440c276321099799bcd2d9c142a883380f20a42977f663 Sep 06 19:43:57.767311 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:57.767276 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97ad1dea-1042-4754-b5f9-6adcfe3840c3" path="/var/lib/kubelet/pods/97ad1dea-1042-4754-b5f9-6adcfe3840c3/volumes" Sep 06 19:43:57.881376 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:57.881337 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" event={"ID":"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5","Type":"ContainerStarted","Data":"7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd"} Sep 06 19:43:57.881582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:57.881409 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:43:57.881582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:57.881435 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" event={"ID":"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5","Type":"ContainerStarted","Data":"45445e8a6e258633a9440c276321099799bcd2d9c142a883380f20a42977f663"} Sep 06 19:43:57.896096 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:43:57.896026 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" podStartSLOduration=1.896012808 podStartE2EDuration="1.896012808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:43:57.895874004 +0000 UTC m=+980.630478735" watchObservedRunningTime="2026-09-06 19:43:57.896012808 +0000 UTC m=+980.630617571" Sep 06 19:44:08.890990 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:44:08.890913 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:45:00.137848 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.137812 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw"] Sep 06 19:45:00.142710 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.142692 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2"] Sep 06 19:45:00.142853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.142833 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" Sep 06 19:45:00.145059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.145038 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:45:00.145925 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.145910 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" Sep 06 19:45:00.150633 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.150606 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw"] Sep 06 19:45:00.154317 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.154292 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2"] Sep 06 19:45:00.155781 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.155764 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" Sep 06 19:45:00.164467 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.164446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" Sep 06 19:45:00.448788 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.448741 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2"] Sep 06 19:45:00.450852 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:45:00.450824 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c73a80e_61d1_4dd6_8314_18ff76293be8.slice/crio-8970864a33163d2334726553f42223f2d7b622e2a3dbf424e5609906a459c979 WatchSource:0}: Error finding container 8970864a33163d2334726553f42223f2d7b622e2a3dbf424e5609906a459c979: Status 404 returned error can't find the container with id 8970864a33163d2334726553f42223f2d7b622e2a3dbf424e5609906a459c979 Sep 06 19:45:00.453012 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.452994 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 19:45:00.453729 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:00.453691 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw"] Sep 06 19:45:00.458573 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:45:00.458437 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c2df99_f822_47b7_8bfa_fe10008f7e7d.slice/crio-6c4cb6350202c34e9323c5acbc5726ba09f3214a7c6e87b65f319cc5fc0d3c22 WatchSource:0}: Error finding container 6c4cb6350202c34e9323c5acbc5726ba09f3214a7c6e87b65f319cc5fc0d3c22: Status 404 returned error can't find the container with id 6c4cb6350202c34e9323c5acbc5726ba09f3214a7c6e87b65f319cc5fc0d3c22 Sep 06 19:45:01.247261 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:01.247223 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" event={"ID":"4c73a80e-61d1-4dd6-8314-18ff76293be8","Type":"ContainerStarted","Data":"8970864a33163d2334726553f42223f2d7b622e2a3dbf424e5609906a459c979"} Sep 06 19:45:01.248792 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:01.248742 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" event={"ID":"56c2df99-f822-47b7-8bfa-fe10008f7e7d","Type":"ContainerStarted","Data":"6c4cb6350202c34e9323c5acbc5726ba09f3214a7c6e87b65f319cc5fc0d3c22"} Sep 06 19:45:03.150902 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.150819 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh"] Sep 06 19:45:03.154452 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.154435 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:03.156758 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.156729 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:45:03.163324 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.163298 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh"] Sep 06 19:45:03.247129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.247087 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6cec38-e008-4600-a45e-40fb4e3f8ac8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-838def-vmfhh\" (UID: \"9b6cec38-e008-4600-a45e-40fb4e3f8ac8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:03.264538 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.264499 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" event={"ID":"4c73a80e-61d1-4dd6-8314-18ff76293be8","Type":"ContainerStarted","Data":"47528678be9f26f8f8bfcf15a7b799cb6b7f236ec5d64747c91216a80605f4d9"} Sep 06 19:45:03.266299 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.266275 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" event={"ID":"56c2df99-f822-47b7-8bfa-fe10008f7e7d","Type":"ContainerStarted","Data":"4aa87c3646c366b7a14ceb52861ed6d3f23fd571d377636567941421147c2e0e"} Sep 06 19:45:03.278648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.278609 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" podStartSLOduration=1.176128841 podStartE2EDuration="3.278595983s" totalImagesPullingTime="2.102467142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:45:03.277138854 +0000 UTC m=+1046.011743597" watchObservedRunningTime="2026-09-06 19:45:03.278595983 +0000 UTC m=+1046.013200705" Sep 06 19:45:03.291332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.291292 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" podStartSLOduration=1.200631537 podStartE2EDuration="3.291279514s" totalImagesPullingTime="2.090647977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:45:03.290230168 +0000 UTC m=+1046.024834900" watchObservedRunningTime="2026-09-06 19:45:03.291279514 +0000 UTC m=+1046.025884244" Sep 06 19:45:03.347928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.347894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6cec38-e008-4600-a45e-40fb4e3f8ac8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-838def-vmfhh\" (UID: \"9b6cec38-e008-4600-a45e-40fb4e3f8ac8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:03.348490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.348470 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6cec38-e008-4600-a45e-40fb4e3f8ac8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-838def-vmfhh\" (UID: \"9b6cec38-e008-4600-a45e-40fb4e3f8ac8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:03.465586 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.465553 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:03.736659 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:03.736634 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh"] Sep 06 19:45:03.738612 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:45:03.738585 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6cec38_e008_4600_a45e_40fb4e3f8ac8.slice/crio-7c0a43f6487cdfe0ea18706dfe0c038b70ed9eadab1f561fa8ece76db16e6906 WatchSource:0}: Error finding container 7c0a43f6487cdfe0ea18706dfe0c038b70ed9eadab1f561fa8ece76db16e6906: Status 404 returned error can't find the container with id 7c0a43f6487cdfe0ea18706dfe0c038b70ed9eadab1f561fa8ece76db16e6906 Sep 06 19:45:04.273261 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.273175 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" event={"ID":"9b6cec38-e008-4600-a45e-40fb4e3f8ac8","Type":"ContainerStarted","Data":"94f9e51b7373184aa9f42758dc35d0489cfcaf790ac7a55cb737535e5343b21d"} Sep 06 19:45:04.273261 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.273227 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" event={"ID":"9b6cec38-e008-4600-a45e-40fb4e3f8ac8","Type":"ContainerStarted","Data":"7c0a43f6487cdfe0ea18706dfe0c038b70ed9eadab1f561fa8ece76db16e6906"} Sep 06 19:45:04.274875 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.274846 2575 generic.go:356] "Generic (PLEG): container finished" podID="4c73a80e-61d1-4dd6-8314-18ff76293be8" containerID="47528678be9f26f8f8bfcf15a7b799cb6b7f236ec5d64747c91216a80605f4d9" exitCode=0 Sep 06 19:45:04.275011 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.274883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" event={"ID":"4c73a80e-61d1-4dd6-8314-18ff76293be8","Type":"ContainerDied","Data":"47528678be9f26f8f8bfcf15a7b799cb6b7f236ec5d64747c91216a80605f4d9"} Sep 06 19:45:04.276375 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.276357 2575 generic.go:356] "Generic (PLEG): container finished" podID="56c2df99-f822-47b7-8bfa-fe10008f7e7d" containerID="4aa87c3646c366b7a14ceb52861ed6d3f23fd571d377636567941421147c2e0e" exitCode=0 Sep 06 19:45:04.276463 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.276434 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" event={"ID":"56c2df99-f822-47b7-8bfa-fe10008f7e7d","Type":"ContainerDied","Data":"4aa87c3646c366b7a14ceb52861ed6d3f23fd571d377636567941421147c2e0e"} Sep 06 19:45:04.292004 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:04.291966 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" podStartSLOduration=1.291954894 podStartE2EDuration="1.291954894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:45:04.28871999 +0000 UTC m=+1047.023324723" watchObservedRunningTime="2026-09-06 19:45:04.291954894 +0000 UTC m=+1047.026559627" Sep 06 19:45:05.292345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:05.292302 2575 generic.go:356] "Generic (PLEG): container finished" podID="9b6cec38-e008-4600-a45e-40fb4e3f8ac8" containerID="94f9e51b7373184aa9f42758dc35d0489cfcaf790ac7a55cb737535e5343b21d" exitCode=0 Sep 06 19:45:05.292732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:05.292389 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" event={"ID":"9b6cec38-e008-4600-a45e-40fb4e3f8ac8","Type":"ContainerDied","Data":"94f9e51b7373184aa9f42758dc35d0489cfcaf790ac7a55cb737535e5343b21d"} Sep 06 19:45:05.431113 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:05.431079 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" Sep 06 19:45:05.471539 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:05.471512 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" Sep 06 19:45:06.299534 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.299506 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" Sep 06 19:45:06.300053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.299504 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2" event={"ID":"4c73a80e-61d1-4dd6-8314-18ff76293be8","Type":"ContainerDied","Data":"8970864a33163d2334726553f42223f2d7b622e2a3dbf424e5609906a459c979"} Sep 06 19:45:06.300053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.299637 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8970864a33163d2334726553f42223f2d7b622e2a3dbf424e5609906a459c979" Sep 06 19:45:06.301025 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.301008 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" Sep 06 19:45:06.301146 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.301039 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw" event={"ID":"56c2df99-f822-47b7-8bfa-fe10008f7e7d","Type":"ContainerDied","Data":"6c4cb6350202c34e9323c5acbc5726ba09f3214a7c6e87b65f319cc5fc0d3c22"} Sep 06 19:45:06.301146 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.301078 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c4cb6350202c34e9323c5acbc5726ba09f3214a7c6e87b65f319cc5fc0d3c22" Sep 06 19:45:06.437015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.436985 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:06.476533 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.476507 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6cec38-e008-4600-a45e-40fb4e3f8ac8-maas-api-service-ca\") pod \"9b6cec38-e008-4600-a45e-40fb4e3f8ac8\" (UID: \"9b6cec38-e008-4600-a45e-40fb4e3f8ac8\") " Sep 06 19:45:06.476832 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.476812 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6cec38-e008-4600-a45e-40fb4e3f8ac8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b6cec38-e008-4600-a45e-40fb4e3f8ac8" (UID: "9b6cec38-e008-4600-a45e-40fb4e3f8ac8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:45:06.577554 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:06.577462 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6cec38-e008-4600-a45e-40fb4e3f8ac8-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:07.307732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:07.307694 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" event={"ID":"9b6cec38-e008-4600-a45e-40fb4e3f8ac8","Type":"ContainerDied","Data":"7c0a43f6487cdfe0ea18706dfe0c038b70ed9eadab1f561fa8ece76db16e6906"} Sep 06 19:45:07.307732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:07.307723 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh" Sep 06 19:45:07.308188 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:07.307744 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c0a43f6487cdfe0ea18706dfe0c038b70ed9eadab1f561fa8ece76db16e6906" Sep 06 19:45:13.674170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.674136 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw"] Sep 06 19:45:13.676398 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.676374 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-838def-29812065-5rscw"] Sep 06 19:45:13.765107 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.765073 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56c2df99-f822-47b7-8bfa-fe10008f7e7d" path="/var/lib/kubelet/pods/56c2df99-f822-47b7-8bfa-fe10008f7e7d/volumes" Sep 06 19:45:13.765340 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.765323 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk"] Sep 06 19:45:13.765722 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.765692 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" podUID="82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" containerName="payload-pre-processing" containerID="cri-o://7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd" gracePeriod=30 Sep 06 19:45:13.936227 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.936135 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh"] Sep 06 19:45:13.938911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:13.938870 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-838def-vmfhh"] Sep 06 19:45:14.029026 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.028975 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:45:14.149784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.149719 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vgqf\" (UniqueName: \"kubernetes.io/projected/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-kube-api-access-6vgqf\") pod \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " Sep 06 19:45:14.149968 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.149835 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-plugins-config-volume\") pod \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\" (UID: \"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5\") " Sep 06 19:45:14.150236 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.150204 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" (UID: "82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:45:14.152084 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.152062 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-kube-api-access-6vgqf" (OuterVolumeSpecName: "kube-api-access-6vgqf") pod "82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" (UID: "82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5"). InnerVolumeSpecName "kube-api-access-6vgqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:45:14.251262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.251167 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vgqf\" (UniqueName: \"kubernetes.io/projected/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-kube-api-access-6vgqf\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:14.251262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.251208 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:14.354082 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.354048 2575 generic.go:356] "Generic (PLEG): container finished" podID="82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" containerID="7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd" exitCode=0 Sep 06 19:45:14.354285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.354094 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" Sep 06 19:45:14.354285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.354139 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" event={"ID":"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5","Type":"ContainerDied","Data":"7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd"} Sep 06 19:45:14.354285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.354200 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk" event={"ID":"82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5","Type":"ContainerDied","Data":"45445e8a6e258633a9440c276321099799bcd2d9c142a883380f20a42977f663"} Sep 06 19:45:14.354285 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.354229 2575 scope.go:117] "RemoveContainer" containerID="7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd" Sep 06 19:45:14.363352 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.363338 2575 scope.go:117] "RemoveContainer" containerID="7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd" Sep 06 19:45:14.363604 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:45:14.363588 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd\": container with ID starting with 7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd not found: ID does not exist" containerID="7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd" Sep 06 19:45:14.363651 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.363613 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd"} err="failed to get container status \"7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd\": rpc error: code = NotFound desc = could not find container \"7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd\": container with ID starting with 7148e602e50dc76b0f4bda4c0c4e1d390dcfdb896767361157b8ee2d574c54fd not found: ID does not exist" Sep 06 19:45:14.376118 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.376088 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk"] Sep 06 19:45:14.378983 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:14.378964 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-838def-5cd74f7ff4-6vmfk"] Sep 06 19:45:15.762168 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:15.762133 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" path="/var/lib/kubelet/pods/82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5/volumes" Sep 06 19:45:15.762522 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:15.762452 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6cec38-e008-4600-a45e-40fb4e3f8ac8" path="/var/lib/kubelet/pods/9b6cec38-e008-4600-a45e-40fb4e3f8ac8/volumes" Sep 06 19:45:20.947352 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947319 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc"] Sep 06 19:45:20.947775 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947738 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c73a80e-61d1-4dd6-8314-18ff76293be8" containerName="cleanup" Sep 06 19:45:20.947775 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947774 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c73a80e-61d1-4dd6-8314-18ff76293be8" containerName="cleanup" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947788 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6cec38-e008-4600-a45e-40fb4e3f8ac8" containerName="revoke-keys" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947796 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6cec38-e008-4600-a45e-40fb4e3f8ac8" containerName="revoke-keys" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947818 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" containerName="payload-pre-processing" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947825 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" containerName="payload-pre-processing" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947842 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56c2df99-f822-47b7-8bfa-fe10008f7e7d" containerName="cleanup" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947849 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="56c2df99-f822-47b7-8bfa-fe10008f7e7d" containerName="cleanup" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947945 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c73a80e-61d1-4dd6-8314-18ff76293be8" containerName="cleanup" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947959 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="56c2df99-f822-47b7-8bfa-fe10008f7e7d" containerName="cleanup" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947970 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="82c93023-4b6d-46d9-b6a4-5d6c5d3b27e5" containerName="payload-pre-processing" Sep 06 19:45:20.947980 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.947981 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6cec38-e008-4600-a45e-40fb4e3f8ac8" containerName="revoke-keys" Sep 06 19:45:20.952422 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.952403 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:20.954599 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.954575 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-2c0c66-openshift-default-dockercfg-vpjnt\"" Sep 06 19:45:20.962475 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:20.962447 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc"] Sep 06 19:45:21.113488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113450 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-certs\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.113488 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-token\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.113814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113509 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-data\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.113814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113610 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380a2dfe-c0d1-469e-893a-6ad22d166f86-istiod-ca-cert\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.113814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113650 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-socket\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.113814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2mq2\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-kube-api-access-c2mq2\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.113814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113794 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-credential-socket\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.114083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-envoy\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.114083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.113851 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-podinfo\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.214957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.214870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-certs\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.214957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.214909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-token\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.214957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.214928 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-data\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.214964 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380a2dfe-c0d1-469e-893a-6ad22d166f86-istiod-ca-cert\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.214988 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-socket\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215014 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2mq2\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-kube-api-access-c2mq2\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215041 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-credential-socket\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215065 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-envoy\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-podinfo\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215492 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215325 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-certs\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215510 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-data\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-socket\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215619 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215602 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-credential-socket\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.215730 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.215708 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380a2dfe-c0d1-469e-893a-6ad22d166f86-istiod-ca-cert\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.217555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.217524 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-envoy\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.217732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.217717 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-podinfo\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.222658 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.222633 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-token\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.222822 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.222808 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2mq2\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-kube-api-access-c2mq2\") pod \"e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.263859 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.263823 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:21.540239 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.540210 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc"] Sep 06 19:45:21.542230 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:45:21.542202 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380a2dfe_c0d1_469e_893a_6ad22d166f86.slice/crio-926446467b461307860673bfc6c584d4e57c4c5572720d2b8bb1bcfb637c443c WatchSource:0}: Error finding container 926446467b461307860673bfc6c584d4e57c4c5572720d2b8bb1bcfb637c443c: Status 404 returned error can't find the container with id 926446467b461307860673bfc6c584d4e57c4c5572720d2b8bb1bcfb637c443c Sep 06 19:45:21.544736 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.544695 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:45:21.544841 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.544807 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:45:21.544877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:21.544862 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:45:22.401817 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:22.401776 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" event={"ID":"380a2dfe-c0d1-469e-893a-6ad22d166f86","Type":"ContainerStarted","Data":"b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6"} Sep 06 19:45:22.402182 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:22.401833 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" event={"ID":"380a2dfe-c0d1-469e-893a-6ad22d166f86","Type":"ContainerStarted","Data":"926446467b461307860673bfc6c584d4e57c4c5572720d2b8bb1bcfb637c443c"} Sep 06 19:45:22.421579 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:22.421528 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" podStartSLOduration=2.421512218 podStartE2EDuration="2.421512218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:45:22.418852495 +0000 UTC m=+1065.153457225" watchObservedRunningTime="2026-09-06 19:45:22.421512218 +0000 UTC m=+1065.156116948" Sep 06 19:45:23.264190 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:23.264154 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:23.269278 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:23.269251 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:23.408009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:23.407979 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:23.408838 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:23.408818 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:50.738143 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:50.738107 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc"] Sep 06 19:45:50.738591 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:50.738354 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" podUID="380a2dfe-c0d1-469e-893a-6ad22d166f86" containerName="istio-proxy" containerID="cri-o://b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6" gracePeriod=30 Sep 06 19:45:52.046479 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.046445 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz"] Sep 06 19:45:52.051336 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.051320 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:52.053805 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.053774 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:45:52.053943 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.053788 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:45:52.056134 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.056107 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz"] Sep 06 19:45:52.103911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.103877 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2858e7b-1644-4660-9c37-07f6041fa541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz\" (UID: \"e2858e7b-1644-4660-9c37-07f6041fa541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:52.205102 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.205068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2858e7b-1644-4660-9c37-07f6041fa541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz\" (UID: \"e2858e7b-1644-4660-9c37-07f6041fa541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:52.205717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.205690 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2858e7b-1644-4660-9c37-07f6041fa541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz\" (UID: \"e2858e7b-1644-4660-9c37-07f6041fa541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:52.363992 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.363907 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:52.635166 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:52.635139 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz"] Sep 06 19:45:52.636626 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:45:52.636581 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2858e7b_1644_4660_9c37_07f6041fa541.slice/crio-11be078ac2b56d1964b956695174a3e9e83a59d720764d3b3237a29d5c32691e WatchSource:0}: Error finding container 11be078ac2b56d1964b956695174a3e9e83a59d720764d3b3237a29d5c32691e: Status 404 returned error can't find the container with id 11be078ac2b56d1964b956695174a3e9e83a59d720764d3b3237a29d5c32691e Sep 06 19:45:53.410165 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:53.410134 2575 patch_prober.go:28] interesting pod/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 06 19:45:53.410498 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:53.410188 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" podUID="380a2dfe-c0d1-469e-893a-6ad22d166f86" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 06 19:45:53.577378 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:53.577280 2575 generic.go:356] "Generic (PLEG): container finished" podID="e2858e7b-1644-4660-9c37-07f6041fa541" containerID="abd637c13469d5996e02bdfaac4f2d1232a5f34999634b95c1fa3cfeb1b74320" exitCode=0 Sep 06 19:45:53.577546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:53.577368 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" event={"ID":"e2858e7b-1644-4660-9c37-07f6041fa541","Type":"ContainerDied","Data":"abd637c13469d5996e02bdfaac4f2d1232a5f34999634b95c1fa3cfeb1b74320"} Sep 06 19:45:53.577546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:53.577419 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" event={"ID":"e2858e7b-1644-4660-9c37-07f6041fa541","Type":"ContainerStarted","Data":"11be078ac2b56d1964b956695174a3e9e83a59d720764d3b3237a29d5c32691e"} Sep 06 19:45:54.723925 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:54.723897 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:54.828627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:54.828596 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2858e7b-1644-4660-9c37-07f6041fa541-maas-api-service-ca\") pod \"e2858e7b-1644-4660-9c37-07f6041fa541\" (UID: \"e2858e7b-1644-4660-9c37-07f6041fa541\") " Sep 06 19:45:54.829029 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:54.829005 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2858e7b-1644-4660-9c37-07f6041fa541-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2858e7b-1644-4660-9c37-07f6041fa541" (UID: "e2858e7b-1644-4660-9c37-07f6041fa541"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:45:54.829153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:54.829124 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2858e7b-1644-4660-9c37-07f6041fa541-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:55.589240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:55.589210 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" Sep 06 19:45:55.589240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:55.589231 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz" event={"ID":"e2858e7b-1644-4660-9c37-07f6041fa541","Type":"ContainerDied","Data":"11be078ac2b56d1964b956695174a3e9e83a59d720764d3b3237a29d5c32691e"} Sep 06 19:45:55.589467 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:55.589275 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11be078ac2b56d1964b956695174a3e9e83a59d720764d3b3237a29d5c32691e" Sep 06 19:45:55.990730 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:55.990707 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:56.140086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140002 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-data\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140040 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2mq2\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-kube-api-access-c2mq2\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140105 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-socket\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140124 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-podinfo\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140140 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-certs\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140169 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-credential-socket\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140204 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-envoy\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140254 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-token\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140289 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380a2dfe-c0d1-469e-893a-6ad22d166f86-istiod-ca-cert\") pod \"380a2dfe-c0d1-469e-893a-6ad22d166f86\" (UID: \"380a2dfe-c0d1-469e-893a-6ad22d166f86\") " Sep 06 19:45:56.140344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140329 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-data" (OuterVolumeSpecName: "istio-data") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:45:56.140779 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140495 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:45:56.140779 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140564 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:45:56.140779 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140587 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:45:56.140779 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140649 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.140779 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140671 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-data\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.141013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.140804 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380a2dfe-c0d1-469e-893a-6ad22d166f86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:45:56.142647 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.142624 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-token" (OuterVolumeSpecName: "istio-token") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:45:56.142798 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.142663 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 19:45:56.143074 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.143051 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:45:56.143118 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.143054 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-kube-api-access-c2mq2" (OuterVolumeSpecName: "kube-api-access-c2mq2") pod "380a2dfe-c0d1-469e-893a-6ad22d166f86" (UID: "380a2dfe-c0d1-469e-893a-6ad22d166f86"). InnerVolumeSpecName "kube-api-access-c2mq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:45:56.242131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242093 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-token\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.242131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242122 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380a2dfe-c0d1-469e-893a-6ad22d166f86-istiod-ca-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.242131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242131 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2mq2\" (UniqueName: \"kubernetes.io/projected/380a2dfe-c0d1-469e-893a-6ad22d166f86-kube-api-access-c2mq2\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.242131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242140 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-podinfo\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.242396 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242149 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-workload-certs\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.242396 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242157 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-credential-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.242396 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.242166 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380a2dfe-c0d1-469e-893a-6ad22d166f86-istio-envoy\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:45:56.595811 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.595737 2575 generic.go:356] "Generic (PLEG): container finished" podID="380a2dfe-c0d1-469e-893a-6ad22d166f86" containerID="b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6" exitCode=0 Sep 06 19:45:56.595811 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.595803 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" Sep 06 19:45:56.596022 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.595821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" event={"ID":"380a2dfe-c0d1-469e-893a-6ad22d166f86","Type":"ContainerDied","Data":"b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6"} Sep 06 19:45:56.596022 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.595877 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc" event={"ID":"380a2dfe-c0d1-469e-893a-6ad22d166f86","Type":"ContainerDied","Data":"926446467b461307860673bfc6c584d4e57c4c5572720d2b8bb1bcfb637c443c"} Sep 06 19:45:56.596022 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.595902 2575 scope.go:117] "RemoveContainer" containerID="b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6" Sep 06 19:45:56.606760 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.606730 2575 scope.go:117] "RemoveContainer" containerID="b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6" Sep 06 19:45:56.607053 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:45:56.607029 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6\": container with ID starting with b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6 not found: ID does not exist" containerID="b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6" Sep 06 19:45:56.607120 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.607065 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6"} err="failed to get container status \"b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6\": rpc error: code = NotFound desc = could not find container \"b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6\": container with ID starting with b490041c4c99c9c55ae1e3979c975a0b8fa8376fa487a4cae73d07cc73196fb6 not found: ID does not exist" Sep 06 19:45:56.627666 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.627637 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc"] Sep 06 19:45:56.629798 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:56.629771 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-2c0c66-openshift-default-56c4b5897b-df2pc"] Sep 06 19:45:57.762551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:57.762518 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="380a2dfe-c0d1-469e-893a-6ad22d166f86" path="/var/lib/kubelet/pods/380a2dfe-c0d1-469e-893a-6ad22d166f86/volumes" Sep 06 19:45:59.337035 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337002 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn"] Sep 06 19:45:59.337427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337373 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2858e7b-1644-4660-9c37-07f6041fa541" containerName="revoke-keys" Sep 06 19:45:59.337427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337385 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2858e7b-1644-4660-9c37-07f6041fa541" containerName="revoke-keys" Sep 06 19:45:59.337427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337403 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380a2dfe-c0d1-469e-893a-6ad22d166f86" containerName="istio-proxy" Sep 06 19:45:59.337427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337408 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="380a2dfe-c0d1-469e-893a-6ad22d166f86" containerName="istio-proxy" Sep 06 19:45:59.337564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337469 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2858e7b-1644-4660-9c37-07f6041fa541" containerName="revoke-keys" Sep 06 19:45:59.337564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.337480 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="380a2dfe-c0d1-469e-893a-6ad22d166f86" containerName="istio-proxy" Sep 06 19:45:59.340422 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.340405 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:45:59.342782 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.342731 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:45:59.342782 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.342774 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:45:59.348086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.348058 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn"] Sep 06 19:45:59.473683 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.473647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0327006f-4daf-4870-b6e4-f3df6a0e01b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn\" (UID: \"0327006f-4daf-4870-b6e4-f3df6a0e01b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:45:59.574582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.574541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0327006f-4daf-4870-b6e4-f3df6a0e01b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn\" (UID: \"0327006f-4daf-4870-b6e4-f3df6a0e01b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:45:59.575150 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.575130 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0327006f-4daf-4870-b6e4-f3df6a0e01b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn\" (UID: \"0327006f-4daf-4870-b6e4-f3df6a0e01b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:45:59.652927 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.652846 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:45:59.922251 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:45:59.922228 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn"] Sep 06 19:45:59.924221 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:45:59.924104 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0327006f_4daf_4870_b6e4_f3df6a0e01b9.slice/crio-491bb7f343b7e620fb180066930f5f8f8697e5bc7a974830a115a2f91ea9f509 WatchSource:0}: Error finding container 491bb7f343b7e620fb180066930f5f8f8697e5bc7a974830a115a2f91ea9f509: Status 404 returned error can't find the container with id 491bb7f343b7e620fb180066930f5f8f8697e5bc7a974830a115a2f91ea9f509 Sep 06 19:46:00.621004 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:00.620964 2575 generic.go:356] "Generic (PLEG): container finished" podID="0327006f-4daf-4870-b6e4-f3df6a0e01b9" containerID="550b11cac4005694de047accafcf927e60d321cd8c137f5171d86db5c433e1c8" exitCode=0 Sep 06 19:46:00.621434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:00.621041 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" event={"ID":"0327006f-4daf-4870-b6e4-f3df6a0e01b9","Type":"ContainerDied","Data":"550b11cac4005694de047accafcf927e60d321cd8c137f5171d86db5c433e1c8"} Sep 06 19:46:00.621434 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:00.621093 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" event={"ID":"0327006f-4daf-4870-b6e4-f3df6a0e01b9","Type":"ContainerStarted","Data":"491bb7f343b7e620fb180066930f5f8f8697e5bc7a974830a115a2f91ea9f509"} Sep 06 19:46:01.758012 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:01.757990 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:46:01.899994 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:01.899894 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0327006f-4daf-4870-b6e4-f3df6a0e01b9-maas-api-service-ca\") pod \"0327006f-4daf-4870-b6e4-f3df6a0e01b9\" (UID: \"0327006f-4daf-4870-b6e4-f3df6a0e01b9\") " Sep 06 19:46:01.900300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:01.900272 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0327006f-4daf-4870-b6e4-f3df6a0e01b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0327006f-4daf-4870-b6e4-f3df6a0e01b9" (UID: "0327006f-4daf-4870-b6e4-f3df6a0e01b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:46:02.001015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.000981 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0327006f-4daf-4870-b6e4-f3df6a0e01b9-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:02.633893 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.633851 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" event={"ID":"0327006f-4daf-4870-b6e4-f3df6a0e01b9","Type":"ContainerDied","Data":"491bb7f343b7e620fb180066930f5f8f8697e5bc7a974830a115a2f91ea9f509"} Sep 06 19:46:02.633893 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.633885 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn" Sep 06 19:46:02.634109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.633945 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="491bb7f343b7e620fb180066930f5f8f8697e5bc7a974830a115a2f91ea9f509" Sep 06 19:46:02.776815 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.776785 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh"] Sep 06 19:46:02.777268 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.777057 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" podUID="3311e9b7-16ec-45b2-9b6e-b20481820681" containerName="payload-pre-processing" containerID="cri-o://8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6" gracePeriod=30 Sep 06 19:46:02.922177 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.922143 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz"] Sep 06 19:46:02.925801 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:02.925765 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6433a6-zwkxz"] Sep 06 19:46:03.038555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.038529 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:46:03.214383 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.214350 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shhmb\" (UniqueName: \"kubernetes.io/projected/3311e9b7-16ec-45b2-9b6e-b20481820681-kube-api-access-shhmb\") pod \"3311e9b7-16ec-45b2-9b6e-b20481820681\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " Sep 06 19:46:03.214559 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.214437 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3311e9b7-16ec-45b2-9b6e-b20481820681-plugins-config-volume\") pod \"3311e9b7-16ec-45b2-9b6e-b20481820681\" (UID: \"3311e9b7-16ec-45b2-9b6e-b20481820681\") " Sep 06 19:46:03.214823 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.214800 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3311e9b7-16ec-45b2-9b6e-b20481820681-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3311e9b7-16ec-45b2-9b6e-b20481820681" (UID: "3311e9b7-16ec-45b2-9b6e-b20481820681"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:46:03.216598 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.216569 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3311e9b7-16ec-45b2-9b6e-b20481820681-kube-api-access-shhmb" (OuterVolumeSpecName: "kube-api-access-shhmb") pod "3311e9b7-16ec-45b2-9b6e-b20481820681" (UID: "3311e9b7-16ec-45b2-9b6e-b20481820681"). InnerVolumeSpecName "kube-api-access-shhmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:46:03.315721 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.315684 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shhmb\" (UniqueName: \"kubernetes.io/projected/3311e9b7-16ec-45b2-9b6e-b20481820681-kube-api-access-shhmb\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:03.315721 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.315713 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3311e9b7-16ec-45b2-9b6e-b20481820681-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:03.641541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.641454 2575 generic.go:356] "Generic (PLEG): container finished" podID="3311e9b7-16ec-45b2-9b6e-b20481820681" containerID="8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6" exitCode=0 Sep 06 19:46:03.641541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.641497 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" Sep 06 19:46:03.641715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.641541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" event={"ID":"3311e9b7-16ec-45b2-9b6e-b20481820681","Type":"ContainerDied","Data":"8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6"} Sep 06 19:46:03.641715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.641598 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh" event={"ID":"3311e9b7-16ec-45b2-9b6e-b20481820681","Type":"ContainerDied","Data":"dea3ff6cdce8216f890339ef182a9bec2feee6d1190ccdf4f0ee9761f1b18fb7"} Sep 06 19:46:03.641715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.641623 2575 scope.go:117] "RemoveContainer" containerID="8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6" Sep 06 19:46:03.651148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.651128 2575 scope.go:117] "RemoveContainer" containerID="8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6" Sep 06 19:46:03.651444 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:46:03.651423 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6\": container with ID starting with 8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6 not found: ID does not exist" containerID="8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6" Sep 06 19:46:03.651491 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.651453 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6"} err="failed to get container status \"8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6\": rpc error: code = NotFound desc = could not find container \"8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6\": container with ID starting with 8d219c44e79712dec035751be524b7fffaea78f3361f34dc589e60546481eea6 not found: ID does not exist" Sep 06 19:46:03.661709 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.661689 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh"] Sep 06 19:46:03.665226 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.665206 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6433a6-5899ff9ff4-btshh"] Sep 06 19:46:03.763216 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.763183 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3311e9b7-16ec-45b2-9b6e-b20481820681" path="/var/lib/kubelet/pods/3311e9b7-16ec-45b2-9b6e-b20481820681/volumes" Sep 06 19:46:03.763517 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:03.763504 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2858e7b-1644-4660-9c37-07f6041fa541" path="/var/lib/kubelet/pods/e2858e7b-1644-4660-9c37-07f6041fa541/volumes" Sep 06 19:46:09.311389 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:09.311353 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq"] Sep 06 19:46:09.312083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:09.311563 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" containerID="cri-o://f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8" gracePeriod=30 Sep 06 19:46:10.951685 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:10.951652 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn"] Sep 06 19:46:10.954055 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:10.954028 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eb3b3780-drbfn"] Sep 06 19:46:11.762644 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:11.762610 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0327006f-4daf-4870-b6e4-f3df6a0e01b9" path="/var/lib/kubelet/pods/0327006f-4daf-4870-b6e4-f3df6a0e01b9/volumes" Sep 06 19:46:14.565843 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.565816 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:46:14.612824 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612719 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-data\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.612824 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612769 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-certs\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612853 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-socket\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612898 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-credential-socket\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612932 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istiod-ca-cert\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612962 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-envoy\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612945 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612982 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.612998 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh4z4\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-kube-api-access-gh4z4\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613029 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-podinfo\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613055 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-token\") pod \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\" (UID: \"9bd25204-c6e7-43ac-a006-63c40de8f0d7\") " Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613110 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613139 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613286 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613412 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-data\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613435 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-certs\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613450 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-workload-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613463 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-credential-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.613553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.613477 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istiod-ca-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.615292 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.615265 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 19:46:14.615440 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.615353 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:46:14.615643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.615626 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:46:14.615730 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.615661 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-kube-api-access-gh4z4" (OuterVolumeSpecName: "kube-api-access-gh4z4") pod "9bd25204-c6e7-43ac-a006-63c40de8f0d7" (UID: "9bd25204-c6e7-43ac-a006-63c40de8f0d7"). InnerVolumeSpecName "kube-api-access-gh4z4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:46:14.710932 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.710896 2575 generic.go:356] "Generic (PLEG): container finished" podID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerID="f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8" exitCode=0 Sep 06 19:46:14.711124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.710943 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" event={"ID":"9bd25204-c6e7-43ac-a006-63c40de8f0d7","Type":"ContainerDied","Data":"f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8"} Sep 06 19:46:14.711124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.710954 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" Sep 06 19:46:14.711124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.710983 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq" event={"ID":"9bd25204-c6e7-43ac-a006-63c40de8f0d7","Type":"ContainerDied","Data":"812984d18a7967cdd65f9b7dfdebc13582cd1fc8ba20d5b17aae412d491c0abb"} Sep 06 19:46:14.711124 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.711004 2575 scope.go:117] "RemoveContainer" containerID="f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8" Sep 06 19:46:14.714025 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.714005 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-envoy\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.714025 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.714027 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh4z4\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-kube-api-access-gh4z4\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.714162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.714036 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-podinfo\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.714162 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.714048 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bd25204-c6e7-43ac-a006-63c40de8f0d7-istio-token\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:14.720981 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.720962 2575 scope.go:117] "RemoveContainer" containerID="2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a" Sep 06 19:46:14.729265 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.729248 2575 scope.go:117] "RemoveContainer" containerID="f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8" Sep 06 19:46:14.729681 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:46:14.729631 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8\": container with ID starting with f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8 not found: ID does not exist" containerID="f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8" Sep 06 19:46:14.729814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.729672 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8"} err="failed to get container status \"f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8\": rpc error: code = NotFound desc = could not find container \"f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8\": container with ID starting with f2554e3267a6aca3c07f1cdb7f7530844c0251b4815c95c0602819301c4367b8 not found: ID does not exist" Sep 06 19:46:14.729814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.729697 2575 scope.go:117] "RemoveContainer" containerID="2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a" Sep 06 19:46:14.730239 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:46:14.730142 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a\": container with ID starting with 2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a not found: ID does not exist" containerID="2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a" Sep 06 19:46:14.730239 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.730176 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a"} err="failed to get container status \"2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a\": rpc error: code = NotFound desc = could not find container \"2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a\": container with ID starting with 2c318b5721f5fda023169fbb9ee2acc0a2cb900700230877070f8832564c836a not found: ID does not exist" Sep 06 19:46:14.733202 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.733184 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq"] Sep 06 19:46:14.736351 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:14.736328 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6433a6-openshift-default-54798894b8-tqgvq"] Sep 06 19:46:15.761789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:15.761732 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" path="/var/lib/kubelet/pods/9bd25204-c6e7-43ac-a006-63c40de8f0d7/volumes" Sep 06 19:46:20.031092 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.031020 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2"] Sep 06 19:46:20.034474 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.034446 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d5bb2d-29812065-8brf2"] Sep 06 19:46:20.053809 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.053777 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9"] Sep 06 19:46:20.054123 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.054091 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" podUID="c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" containerName="payload-processing" containerID="cri-o://d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc" gracePeriod=30 Sep 06 19:46:20.058110 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.058079 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5"] Sep 06 19:46:20.059100 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.059042 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" podUID="24879275-939e-4705-b536-72f780df3601" containerName="payload-pre-processing" containerID="cri-o://691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3" gracePeriod=30 Sep 06 19:46:20.338880 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.338850 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:46:20.341607 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.341587 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:46:20.365483 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.365455 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22bk4\" (UniqueName: \"kubernetes.io/projected/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-kube-api-access-22bk4\") pod \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " Sep 06 19:46:20.365620 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.365545 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-plugins-config-volume\") pod \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\" (UID: \"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9\") " Sep 06 19:46:20.365620 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.365595 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24879275-939e-4705-b536-72f780df3601-plugins-config-volume\") pod \"24879275-939e-4705-b536-72f780df3601\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " Sep 06 19:46:20.365708 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.365630 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5trl\" (UniqueName: \"kubernetes.io/projected/24879275-939e-4705-b536-72f780df3601-kube-api-access-r5trl\") pod \"24879275-939e-4705-b536-72f780df3601\" (UID: \"24879275-939e-4705-b536-72f780df3601\") " Sep 06 19:46:20.365928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.365903 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" (UID: "c17f9a76-43d1-4691-8d0a-17b6e94fd1b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:46:20.366056 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.365950 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24879275-939e-4705-b536-72f780df3601-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24879275-939e-4705-b536-72f780df3601" (UID: "24879275-939e-4705-b536-72f780df3601"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:46:20.366056 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.366018 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:20.368681 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.368550 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-kube-api-access-22bk4" (OuterVolumeSpecName: "kube-api-access-22bk4") pod "c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" (UID: "c17f9a76-43d1-4691-8d0a-17b6e94fd1b9"). InnerVolumeSpecName "kube-api-access-22bk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:46:20.368681 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.368634 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24879275-939e-4705-b536-72f780df3601-kube-api-access-r5trl" (OuterVolumeSpecName: "kube-api-access-r5trl") pod "24879275-939e-4705-b536-72f780df3601" (UID: "24879275-939e-4705-b536-72f780df3601"). InnerVolumeSpecName "kube-api-access-r5trl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:46:20.466998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.466958 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22bk4\" (UniqueName: \"kubernetes.io/projected/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9-kube-api-access-22bk4\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:20.466998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.466989 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24879275-939e-4705-b536-72f780df3601-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:20.466998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.466997 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5trl\" (UniqueName: \"kubernetes.io/projected/24879275-939e-4705-b536-72f780df3601-kube-api-access-r5trl\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:46:20.747016 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.746980 2575 generic.go:356] "Generic (PLEG): container finished" podID="24879275-939e-4705-b536-72f780df3601" containerID="691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3" exitCode=0 Sep 06 19:46:20.747180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.747026 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" Sep 06 19:46:20.747180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.747031 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" event={"ID":"24879275-939e-4705-b536-72f780df3601","Type":"ContainerDied","Data":"691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3"} Sep 06 19:46:20.747180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.747088 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5" event={"ID":"24879275-939e-4705-b536-72f780df3601","Type":"ContainerDied","Data":"f2834eef568141d737781e9b57742b265f3c28f52299bc205e32155bf5283056"} Sep 06 19:46:20.747180 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.747114 2575 scope.go:117] "RemoveContainer" containerID="691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3" Sep 06 19:46:20.748459 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.748435 2575 generic.go:356] "Generic (PLEG): container finished" podID="c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" containerID="d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc" exitCode=0 Sep 06 19:46:20.748601 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.748464 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" event={"ID":"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9","Type":"ContainerDied","Data":"d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc"} Sep 06 19:46:20.748601 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.748474 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" Sep 06 19:46:20.748601 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.748506 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9" event={"ID":"c17f9a76-43d1-4691-8d0a-17b6e94fd1b9","Type":"ContainerDied","Data":"2f15e19e6ce4f24317227e4410260a84c4edfca976ff40e8491026f869c1986d"} Sep 06 19:46:20.757995 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.757980 2575 scope.go:117] "RemoveContainer" containerID="691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3" Sep 06 19:46:20.758276 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:46:20.758256 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3\": container with ID starting with 691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3 not found: ID does not exist" containerID="691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3" Sep 06 19:46:20.758344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.758288 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3"} err="failed to get container status \"691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3\": rpc error: code = NotFound desc = could not find container \"691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3\": container with ID starting with 691de6fbbbaa9ecdf8f1a9b5de5069e793a016000114f3945e49910a45f23bd3 not found: ID does not exist" Sep 06 19:46:20.758344 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.758305 2575 scope.go:117] "RemoveContainer" containerID="d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc" Sep 06 19:46:20.766796 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.766779 2575 scope.go:117] "RemoveContainer" containerID="d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc" Sep 06 19:46:20.767087 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:46:20.767065 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc\": container with ID starting with d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc not found: ID does not exist" containerID="d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc" Sep 06 19:46:20.767138 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.767099 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc"} err="failed to get container status \"d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc\": rpc error: code = NotFound desc = could not find container \"d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc\": container with ID starting with d6de37ad0b12a2c64db8ce13545a19d04b268c2edc049dbffc3618a4d21f38bc not found: ID does not exist" Sep 06 19:46:20.768247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.768230 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5"] Sep 06 19:46:20.772430 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.772405 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d5bb2d-655df4d798-fgss5"] Sep 06 19:46:20.781928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.781895 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9"] Sep 06 19:46:20.783985 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:20.783963 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d5bb2d-69b9655865-wsxr9"] Sep 06 19:46:21.761051 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:21.761021 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24879275-939e-4705-b536-72f780df3601" path="/var/lib/kubelet/pods/24879275-939e-4705-b536-72f780df3601/volumes" Sep 06 19:46:21.761423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:21.761297 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c73a80e-61d1-4dd6-8314-18ff76293be8" path="/var/lib/kubelet/pods/4c73a80e-61d1-4dd6-8314-18ff76293be8/volumes" Sep 06 19:46:21.761567 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:21.761555 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" path="/var/lib/kubelet/pods/c17f9a76-43d1-4691-8d0a-17b6e94fd1b9/volumes" Sep 06 19:46:27.260393 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260362 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx"] Sep 06 19:46:27.260828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260781 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0327006f-4daf-4870-b6e4-f3df6a0e01b9" containerName="revoke-keys" Sep 06 19:46:27.260828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260795 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0327006f-4daf-4870-b6e4-f3df6a0e01b9" containerName="revoke-keys" Sep 06 19:46:27.260828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260808 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" Sep 06 19:46:27.260828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260815 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" Sep 06 19:46:27.260828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260825 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3311e9b7-16ec-45b2-9b6e-b20481820681" containerName="payload-pre-processing" Sep 06 19:46:27.260828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260831 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3311e9b7-16ec-45b2-9b6e-b20481820681" containerName="payload-pre-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260837 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260842 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260847 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" containerName="payload-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260853 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" containerName="payload-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260858 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24879275-939e-4705-b536-72f780df3601" containerName="payload-pre-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260863 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="24879275-939e-4705-b536-72f780df3601" containerName="payload-pre-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260942 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260954 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="24879275-939e-4705-b536-72f780df3601" containerName="payload-pre-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260963 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0327006f-4daf-4870-b6e4-f3df6a0e01b9" containerName="revoke-keys" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260977 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17f9a76-43d1-4691-8d0a-17b6e94fd1b9" containerName="payload-processing" Sep 06 19:46:27.261009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.260985 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3311e9b7-16ec-45b2-9b6e-b20481820681" containerName="payload-pre-processing" Sep 06 19:46:27.265726 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.265703 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.268339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.268319 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-b571c4-openshift-default-dockercfg-xpq55\"" Sep 06 19:46:27.273843 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.273806 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx"] Sep 06 19:46:27.330398 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-socket\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330398 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330398 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5ffv\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-kube-api-access-n5ffv\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11195ef4-b947-4745-861a-9e9324abd263-istiod-ca-cert\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11195ef4-b947-4745-861a-9e9324abd263-istio-podinfo\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330534 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-data\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-envoy\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330735 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330610 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-certs\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330735 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330631 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-istio-token\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.330735 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.330680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-credential-socket\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431680 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431643 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11195ef4-b947-4745-861a-9e9324abd263-istio-podinfo\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431680 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431685 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-data\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-envoy\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-certs\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431794 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-istio-token\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-credential-socket\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431874 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-socket\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5ffv\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-kube-api-access-n5ffv\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.431948 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.431906 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11195ef4-b947-4745-861a-9e9324abd263-istiod-ca-cert\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.432293 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.432229 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-certs\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.432293 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.432281 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-socket\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.432403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.432312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-credential-socket\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.432463 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.432444 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-data\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.432550 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.432534 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11195ef4-b947-4745-861a-9e9324abd263-istiod-ca-cert\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.434785 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.434742 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-envoy\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.434922 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.434905 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11195ef4-b947-4745-861a-9e9324abd263-istio-podinfo\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.439608 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.439582 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5ffv\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-kube-api-access-n5ffv\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.439703 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.439683 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-istio-token\") pod \"e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.578188 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.578104 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:27.855136 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.855103 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx"] Sep 06 19:46:27.856658 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:46:27.856632 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11195ef4_b947_4745_861a_9e9324abd263.slice/crio-2695ab3084f8a56c8533ca470cdd2e0f9b3f5599e79af4eb7f9d0f4bb7879a59 WatchSource:0}: Error finding container 2695ab3084f8a56c8533ca470cdd2e0f9b3f5599e79af4eb7f9d0f4bb7879a59: Status 404 returned error can't find the container with id 2695ab3084f8a56c8533ca470cdd2e0f9b3f5599e79af4eb7f9d0f4bb7879a59 Sep 06 19:46:27.859477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.859439 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:27.859564 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.859519 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:27.859607 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:27.859561 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:28.798144 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:28.798111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" event={"ID":"11195ef4-b947-4745-861a-9e9324abd263","Type":"ContainerStarted","Data":"95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97"} Sep 06 19:46:28.798530 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:28.798165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" event={"ID":"11195ef4-b947-4745-861a-9e9324abd263","Type":"ContainerStarted","Data":"2695ab3084f8a56c8533ca470cdd2e0f9b3f5599e79af4eb7f9d0f4bb7879a59"} Sep 06 19:46:28.817258 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:28.817208 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podStartSLOduration=1.817194412 podStartE2EDuration="1.817194412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:46:28.815660804 +0000 UTC m=+1131.550265559" watchObservedRunningTime="2026-09-06 19:46:28.817194412 +0000 UTC m=+1131.551799142" Sep 06 19:46:29.578584 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:29.578549 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:29.583464 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:29.583440 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:29.803689 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:29.803658 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:29.804812 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:29.804790 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:30.686026 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.685994 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd"] Sep 06 19:46:30.686937 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.686910 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bd25204-c6e7-43ac-a006-63c40de8f0d7" containerName="istio-proxy" Sep 06 19:46:30.690416 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.690395 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.692782 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.692743 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-b12ed2-openshift-default-dockercfg-hlnsm\"" Sep 06 19:46:30.702157 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.702131 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd"] Sep 06 19:46:30.763318 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-socket\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763318 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763320 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-certs\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763360 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-data\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763394 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-credential-socket\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istiod-ca-cert\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-podinfo\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763488 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-envoy\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763555 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-token\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.763801 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.763632 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg856\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-kube-api-access-kg856\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865161 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865126 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-data\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-credential-socket\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istiod-ca-cert\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-podinfo\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-envoy\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865299 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-token\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865420 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg856\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-kube-api-access-kg856\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865520 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-socket\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-certs\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.865627 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865598 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-credential-socket\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.866316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.865981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-socket\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.866316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.866072 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-data\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.866316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.866080 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-certs\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.866316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.866110 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istiod-ca-cert\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.867806 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.867784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-envoy\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.868017 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.867997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-podinfo\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.873485 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.873464 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg856\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-kube-api-access-kg856\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:30.873696 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:30.873677 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-token\") pod \"e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:31.004788 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.004679 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:31.285304 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.285267 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd"] Sep 06 19:46:31.286887 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:46:31.286861 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2480c06_9e85_4eca_98d2_d675c5bcb8dd.slice/crio-085717fba249a9303fd97ea3322e37876d4274490791d02ecfcc7f3945312f61 WatchSource:0}: Error finding container 085717fba249a9303fd97ea3322e37876d4274490791d02ecfcc7f3945312f61: Status 404 returned error can't find the container with id 085717fba249a9303fd97ea3322e37876d4274490791d02ecfcc7f3945312f61 Sep 06 19:46:31.289161 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.289115 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:31.289234 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.289220 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:31.289283 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.289266 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:31.818069 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.817998 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" event={"ID":"c2480c06-9e85-4eca-98d2-d675c5bcb8dd","Type":"ContainerStarted","Data":"c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577"} Sep 06 19:46:31.818069 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.818053 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" event={"ID":"c2480c06-9e85-4eca-98d2-d675c5bcb8dd","Type":"ContainerStarted","Data":"085717fba249a9303fd97ea3322e37876d4274490791d02ecfcc7f3945312f61"} Sep 06 19:46:31.842719 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:31.842668 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podStartSLOduration=1.842652446 podStartE2EDuration="1.842652446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:46:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:46:31.841718765 +0000 UTC m=+1134.576323496" watchObservedRunningTime="2026-09-06 19:46:31.842652446 +0000 UTC m=+1134.577257176" Sep 06 19:46:32.005652 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:32.005621 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:32.010394 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:32.010372 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:32.823364 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:32.823330 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:32.824630 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:32.824607 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:35.744453 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.744415 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk"] Sep 06 19:46:35.747789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.747745 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:35.750149 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.750127 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-b12ed2\"" Sep 06 19:46:35.750263 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.750206 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-b12ed2-dockercfg-xf9z9\"" Sep 06 19:46:35.762809 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.762738 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk"] Sep 06 19:46:35.782974 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.782944 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx"] Sep 06 19:46:35.786295 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.786279 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:35.794693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.794667 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx"] Sep 06 19:46:35.811257 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.811231 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phs7m\" (UniqueName: \"kubernetes.io/projected/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-kube-api-access-phs7m\") pod \"payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:35.811393 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.811327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:35.912588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.912551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkph8\" (UniqueName: \"kubernetes.io/projected/458ec939-67e3-4d12-8800-21245aab56ad-kube-api-access-xkph8\") pod \"payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:35.912800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.912597 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phs7m\" (UniqueName: \"kubernetes.io/projected/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-kube-api-access-phs7m\") pod \"payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:35.912800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.912625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458ec939-67e3-4d12-8800-21245aab56ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:35.912800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.912643 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:35.913426 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.913399 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:35.920005 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:35.919987 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phs7m\" (UniqueName: \"kubernetes.io/projected/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-kube-api-access-phs7m\") pod \"payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:36.013383 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.013295 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkph8\" (UniqueName: \"kubernetes.io/projected/458ec939-67e3-4d12-8800-21245aab56ad-kube-api-access-xkph8\") pod \"payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:36.013383 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.013367 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458ec939-67e3-4d12-8800-21245aab56ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:36.014070 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.014051 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458ec939-67e3-4d12-8800-21245aab56ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:36.020241 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.020220 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkph8\" (UniqueName: \"kubernetes.io/projected/458ec939-67e3-4d12-8800-21245aab56ad-kube-api-access-xkph8\") pod \"payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:36.060131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.060104 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:36.097698 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.097666 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:36.330603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.330570 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk"] Sep 06 19:46:36.332330 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:46:36.332303 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba5dff1_62a4_4fb0_9089_c55d30ec5f44.slice/crio-a8a0631bce3975b2b0b1cbeb2bbd8fef7381bac0c0c493bd278343db2dfcaffb WatchSource:0}: Error finding container a8a0631bce3975b2b0b1cbeb2bbd8fef7381bac0c0c493bd278343db2dfcaffb: Status 404 returned error can't find the container with id a8a0631bce3975b2b0b1cbeb2bbd8fef7381bac0c0c493bd278343db2dfcaffb Sep 06 19:46:36.386035 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.386012 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx"] Sep 06 19:46:36.387704 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:46:36.387679 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458ec939_67e3_4d12_8800_21245aab56ad.slice/crio-b5f694ee53ae868b6b92e28a9ecbf25cd3eab13f0ee325861be4053491112dee WatchSource:0}: Error finding container b5f694ee53ae868b6b92e28a9ecbf25cd3eab13f0ee325861be4053491112dee: Status 404 returned error can't find the container with id b5f694ee53ae868b6b92e28a9ecbf25cd3eab13f0ee325861be4053491112dee Sep 06 19:46:36.856417 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.856376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" event={"ID":"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44","Type":"ContainerStarted","Data":"8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db"} Sep 06 19:46:36.856872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.856577 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:36.856872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.856629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" event={"ID":"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44","Type":"ContainerStarted","Data":"a8a0631bce3975b2b0b1cbeb2bbd8fef7381bac0c0c493bd278343db2dfcaffb"} Sep 06 19:46:36.858049 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.858021 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" event={"ID":"458ec939-67e3-4d12-8800-21245aab56ad","Type":"ContainerStarted","Data":"fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc"} Sep 06 19:46:36.858154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.858073 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:36.858154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.858091 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" event={"ID":"458ec939-67e3-4d12-8800-21245aab56ad","Type":"ContainerStarted","Data":"b5f694ee53ae868b6b92e28a9ecbf25cd3eab13f0ee325861be4053491112dee"} Sep 06 19:46:36.878988 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.872815 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" podStartSLOduration=1.872792224 podStartE2EDuration="1.872792224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:46:36.870666184 +0000 UTC m=+1139.605270916" watchObservedRunningTime="2026-09-06 19:46:36.872792224 +0000 UTC m=+1139.607396955" Sep 06 19:46:36.894251 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:36.894187 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" podStartSLOduration=1.8941666179999999 podStartE2EDuration="1.894166618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:46:36.890916416 +0000 UTC m=+1139.625521147" watchObservedRunningTime="2026-09-06 19:46:36.894166618 +0000 UTC m=+1139.628771351" Sep 06 19:46:45.847928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:45.847872 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 06 19:46:45.848499 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:45.847934 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 06 19:46:46.933856 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:46.933825 2575 generic.go:356] "Generic (PLEG): container finished" podID="11195ef4-b947-4745-861a-9e9324abd263" containerID="95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97" exitCode=0 Sep 06 19:46:46.934382 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:46.933872 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" event={"ID":"11195ef4-b947-4745-861a-9e9324abd263","Type":"ContainerDied","Data":"95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97"} Sep 06 19:46:46.934382 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:46.934313 2575 scope.go:117] "RemoveContainer" containerID="95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97" Sep 06 19:46:46.935382 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:46.935351 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:46.935472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:46.935418 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:46.935472 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:46.935459 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:47.865911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:47.865878 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:46:47.866103 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:47.865957 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:46:47.970195 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:47.970167 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" event={"ID":"11195ef4-b947-4745-861a-9e9324abd263","Type":"ContainerStarted","Data":"b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73"} Sep 06 19:46:48.578570 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:48.578491 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:48.578713 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:48.578677 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Sep 06 19:46:48.578792 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:48.578717 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Sep 06 19:46:49.578560 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:49.578515 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Sep 06 19:46:49.578986 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:49.578589 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Sep 06 19:46:50.579514 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:50.579475 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Sep 06 19:46:50.579926 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:50.579536 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Sep 06 19:46:51.578699 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:51.578666 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Sep 06 19:46:51.578917 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:51.578718 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Sep 06 19:46:51.998246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:51.998212 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerID="c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577" exitCode=0 Sep 06 19:46:51.999802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:51.998353 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" event={"ID":"c2480c06-9e85-4eca-98d2-d675c5bcb8dd","Type":"ContainerDied","Data":"c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577"} Sep 06 19:46:51.999802 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:51.999417 2575 scope.go:117] "RemoveContainer" containerID="c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577" Sep 06 19:46:52.000821 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:52.000787 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:52.001024 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:52.001000 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:52.001176 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:52.001155 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:46:52.578859 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:52.578822 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Sep 06 19:46:52.579032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:52.578875 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Sep 06 19:46:53.015633 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:53.015582 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" event={"ID":"c2480c06-9e85-4eca-98d2-d675c5bcb8dd","Type":"ContainerStarted","Data":"993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399"} Sep 06 19:46:53.681827 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:53.681796 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:53.682278 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:53.682250 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:53.683533 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:53.683508 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:46:54.005906 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:54.005802 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:54.006309 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:54.006272 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 06 19:46:54.006451 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:54.006350 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 06 19:46:55.005886 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:55.005834 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 06 19:46:55.006498 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:55.006455 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 06 19:46:56.006468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:56.006032 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 06 19:46:56.006468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:56.006095 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 06 19:46:57.005856 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:57.005818 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 06 19:46:57.006147 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:57.006117 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 06 19:46:58.005975 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:58.005933 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 06 19:46:58.006355 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:58.006004 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 06 19:46:59.008927 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:59.008856 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:59.009307 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:59.009140 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:46:59.009848 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:46:59.009830 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:47:00.536403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.536369 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd"] Sep 06 19:47:00.539418 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.539396 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.544233 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.544215 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-712616\"" Sep 06 19:47:00.544441 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.544428 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-712616-dockercfg-gvm87\"" Sep 06 19:47:00.550667 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.550642 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd"] Sep 06 19:47:00.574135 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.574108 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q"] Sep 06 19:47:00.576442 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.576427 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.587610 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.587586 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q"] Sep 06 19:47:00.587946 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.587918 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w699f\" (UniqueName: \"kubernetes.io/projected/85d4e583-3537-4daa-baf8-d7df7c6c4449-kube-api-access-w699f\") pod \"payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.588039 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.587950 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d4e583-3537-4daa-baf8-d7df7c6c4449-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.688408 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.688377 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a93431-9d77-4963-b729-887b5c8ab11c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.688602 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.688415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfp5l\" (UniqueName: \"kubernetes.io/projected/a1a93431-9d77-4963-b729-887b5c8ab11c-kube-api-access-tfp5l\") pod \"payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.688602 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.688491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w699f\" (UniqueName: \"kubernetes.io/projected/85d4e583-3537-4daa-baf8-d7df7c6c4449-kube-api-access-w699f\") pod \"payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.688602 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.688520 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d4e583-3537-4daa-baf8-d7df7c6c4449-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.689246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.689227 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d4e583-3537-4daa-baf8-d7df7c6c4449-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.696366 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.696339 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w699f\" (UniqueName: \"kubernetes.io/projected/85d4e583-3537-4daa-baf8-d7df7c6c4449-kube-api-access-w699f\") pod \"payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.790091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.790000 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a93431-9d77-4963-b729-887b5c8ab11c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.790263 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.790135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfp5l\" (UniqueName: \"kubernetes.io/projected/a1a93431-9d77-4963-b729-887b5c8ab11c-kube-api-access-tfp5l\") pod \"payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.790672 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.790646 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a93431-9d77-4963-b729-887b5c8ab11c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.797811 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.797794 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfp5l\" (UniqueName: \"kubernetes.io/projected/a1a93431-9d77-4963-b729-887b5c8ab11c-kube-api-access-tfp5l\") pod \"payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:00.850714 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.850677 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:00.886826 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:00.886789 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:01.124264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:01.124232 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd"] Sep 06 19:47:01.125854 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:47:01.125830 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d4e583_3537_4daa_baf8_d7df7c6c4449.slice/crio-8087b8e035a21d2a10d9be4d41d43a24e5ebfba68aaf3806e6dd72bfec3e19fe WatchSource:0}: Error finding container 8087b8e035a21d2a10d9be4d41d43a24e5ebfba68aaf3806e6dd72bfec3e19fe: Status 404 returned error can't find the container with id 8087b8e035a21d2a10d9be4d41d43a24e5ebfba68aaf3806e6dd72bfec3e19fe Sep 06 19:47:01.192741 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:01.192718 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q"] Sep 06 19:47:01.194432 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:47:01.194407 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a93431_9d77_4963_b729_887b5c8ab11c.slice/crio-7b4f770205515d246047b917f09dd2f7b4b2a8c835a66b2c3e6c3125435beb48 WatchSource:0}: Error finding container 7b4f770205515d246047b917f09dd2f7b4b2a8c835a66b2c3e6c3125435beb48: Status 404 returned error can't find the container with id 7b4f770205515d246047b917f09dd2f7b4b2a8c835a66b2c3e6c3125435beb48 Sep 06 19:47:02.079514 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.079479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" event={"ID":"a1a93431-9d77-4963-b729-887b5c8ab11c","Type":"ContainerStarted","Data":"6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe"} Sep 06 19:47:02.079994 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.079536 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" event={"ID":"a1a93431-9d77-4963-b729-887b5c8ab11c","Type":"ContainerStarted","Data":"7b4f770205515d246047b917f09dd2f7b4b2a8c835a66b2c3e6c3125435beb48"} Sep 06 19:47:02.079994 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.079673 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:02.081318 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.081284 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" event={"ID":"85d4e583-3537-4daa-baf8-d7df7c6c4449","Type":"ContainerStarted","Data":"4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765"} Sep 06 19:47:02.081476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.081336 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" event={"ID":"85d4e583-3537-4daa-baf8-d7df7c6c4449","Type":"ContainerStarted","Data":"8087b8e035a21d2a10d9be4d41d43a24e5ebfba68aaf3806e6dd72bfec3e19fe"} Sep 06 19:47:02.081476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.081382 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:02.095551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.095508 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" podStartSLOduration=2.095496622 podStartE2EDuration="2.095496622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:47:02.093857306 +0000 UTC m=+1164.828462032" watchObservedRunningTime="2026-09-06 19:47:02.095496622 +0000 UTC m=+1164.830101351" Sep 06 19:47:02.111430 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:02.111332 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" podStartSLOduration=2.111308494 podStartE2EDuration="2.111308494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:47:02.107484653 +0000 UTC m=+1164.842089399" watchObservedRunningTime="2026-09-06 19:47:02.111308494 +0000 UTC m=+1164.845913225" Sep 06 19:47:04.597120 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.597086 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll"] Sep 06 19:47:04.600419 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.600404 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.602625 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.602602 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-6ddeb5-dockercfg-bkq2g\"" Sep 06 19:47:04.602904 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.602889 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-6ddeb5\"" Sep 06 19:47:04.612043 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.612020 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll"] Sep 06 19:47:04.732835 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.732794 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25c9977d-ba77-4250-af58-bc7790a66e44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.733009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.732860 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz8fg\" (UniqueName: \"kubernetes.io/projected/25c9977d-ba77-4250-af58-bc7790a66e44-kube-api-access-xz8fg\") pod \"payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.834264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.834228 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25c9977d-ba77-4250-af58-bc7790a66e44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.834507 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.834322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz8fg\" (UniqueName: \"kubernetes.io/projected/25c9977d-ba77-4250-af58-bc7790a66e44-kube-api-access-xz8fg\") pod \"payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.835076 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.835057 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25c9977d-ba77-4250-af58-bc7790a66e44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.841853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.841831 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz8fg\" (UniqueName: \"kubernetes.io/projected/25c9977d-ba77-4250-af58-bc7790a66e44-kube-api-access-xz8fg\") pod \"payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:04.912365 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:04.912283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:05.191084 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:05.191057 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll"] Sep 06 19:47:05.193112 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:47:05.193082 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c9977d_ba77_4250_af58_bc7790a66e44.slice/crio-02d9aa3241e25dd84406a6eb8f7bad8350a145c7dfe3d984f2a3558e087fc01e WatchSource:0}: Error finding container 02d9aa3241e25dd84406a6eb8f7bad8350a145c7dfe3d984f2a3558e087fc01e: Status 404 returned error can't find the container with id 02d9aa3241e25dd84406a6eb8f7bad8350a145c7dfe3d984f2a3558e087fc01e Sep 06 19:47:06.107357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:06.107314 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" event={"ID":"25c9977d-ba77-4250-af58-bc7790a66e44","Type":"ContainerStarted","Data":"b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d"} Sep 06 19:47:06.107728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:06.107378 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" event={"ID":"25c9977d-ba77-4250-af58-bc7790a66e44","Type":"ContainerStarted","Data":"02d9aa3241e25dd84406a6eb8f7bad8350a145c7dfe3d984f2a3558e087fc01e"} Sep 06 19:47:06.107728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:06.107507 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:06.122063 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:06.122021 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" podStartSLOduration=2.122006981 podStartE2EDuration="2.122006981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:47:06.121242246 +0000 UTC m=+1168.855846991" watchObservedRunningTime="2026-09-06 19:47:06.122006981 +0000 UTC m=+1168.856611711" Sep 06 19:47:13.090339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:13.090306 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:47:13.090732 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:13.090386 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:47:17.116353 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:17.116322 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:47:37.763512 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:37.763488 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:47:37.764877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:37.764852 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:47:45.939458 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:45.939418 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr"] Sep 06 19:47:45.943305 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:45.943287 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:45.946075 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:45.946054 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0936e642\"" Sep 06 19:47:45.946196 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:45.946074 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0936e642-dockercfg-9lzs7\"" Sep 06 19:47:45.955254 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:45.955228 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr"] Sep 06 19:47:46.011573 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.011540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjjkz\" (UniqueName: \"kubernetes.io/projected/dd0a7d5b-f327-496f-8113-a1482be79a82-kube-api-access-cjjkz\") pod \"payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.011782 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.011614 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0a7d5b-f327-496f-8113-a1482be79a82-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.112284 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.112250 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0a7d5b-f327-496f-8113-a1482be79a82-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.112468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.112335 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjjkz\" (UniqueName: \"kubernetes.io/projected/dd0a7d5b-f327-496f-8113-a1482be79a82-kube-api-access-cjjkz\") pod \"payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.113114 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.113091 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0a7d5b-f327-496f-8113-a1482be79a82-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.119930 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.119909 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjjkz\" (UniqueName: \"kubernetes.io/projected/dd0a7d5b-f327-496f-8113-a1482be79a82-kube-api-access-cjjkz\") pod \"payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.255205 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.255116 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:46.529272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:46.529246 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr"] Sep 06 19:47:46.530632 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:47:46.530605 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0a7d5b_f327_496f_8113_a1482be79a82.slice/crio-562f448ee8019b1e27aeed8794e6a24af92a30900d8adf149720e879348ae794 WatchSource:0}: Error finding container 562f448ee8019b1e27aeed8794e6a24af92a30900d8adf149720e879348ae794: Status 404 returned error can't find the container with id 562f448ee8019b1e27aeed8794e6a24af92a30900d8adf149720e879348ae794 Sep 06 19:47:47.349378 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:47.349338 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" event={"ID":"dd0a7d5b-f327-496f-8113-a1482be79a82","Type":"ContainerStarted","Data":"2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63"} Sep 06 19:47:47.349789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:47.349410 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" event={"ID":"dd0a7d5b-f327-496f-8113-a1482be79a82","Type":"ContainerStarted","Data":"562f448ee8019b1e27aeed8794e6a24af92a30900d8adf149720e879348ae794"} Sep 06 19:47:47.349789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:47.349437 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:47:47.364684 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:47.364639 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" podStartSLOduration=2.36462414 podStartE2EDuration="2.36462414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:47:47.363385156 +0000 UTC m=+1210.097989885" watchObservedRunningTime="2026-09-06 19:47:47.36462414 +0000 UTC m=+1210.099228871" Sep 06 19:47:58.358956 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:47:58.358926 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:49:54.897015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:54.896975 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk"] Sep 06 19:49:54.900705 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:54.900688 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:54.903324 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:54.903298 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:49:54.903324 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:54.903324 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:49:54.908345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:54.908316 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk"] Sep 06 19:49:54.932974 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:54.932945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/132929b4-dc54-4e2f-bfaf-7c40acf200e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0936e642-sknvk\" (UID: \"132929b4-dc54-4e2f-bfaf-7c40acf200e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:55.034376 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:55.034337 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/132929b4-dc54-4e2f-bfaf-7c40acf200e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0936e642-sknvk\" (UID: \"132929b4-dc54-4e2f-bfaf-7c40acf200e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:55.034989 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:55.034970 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/132929b4-dc54-4e2f-bfaf-7c40acf200e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0936e642-sknvk\" (UID: \"132929b4-dc54-4e2f-bfaf-7c40acf200e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:55.213762 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:55.213711 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:55.486467 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:55.486435 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk"] Sep 06 19:49:55.487715 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:49:55.487686 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132929b4_dc54_4e2f_bfaf_7c40acf200e4.slice/crio-2436205df948adf40b6e480feec34a0d6b0275663f1d504507c800719e436d97 WatchSource:0}: Error finding container 2436205df948adf40b6e480feec34a0d6b0275663f1d504507c800719e436d97: Status 404 returned error can't find the container with id 2436205df948adf40b6e480feec34a0d6b0275663f1d504507c800719e436d97 Sep 06 19:49:56.088761 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:56.088661 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" event={"ID":"132929b4-dc54-4e2f-bfaf-7c40acf200e4","Type":"ContainerStarted","Data":"27c4efbe59f36e07cb5a1afc3a75df73b04d4d1465e9c40e3be6086ca2f90e11"} Sep 06 19:49:56.088761 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:56.088716 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" event={"ID":"132929b4-dc54-4e2f-bfaf-7c40acf200e4","Type":"ContainerStarted","Data":"2436205df948adf40b6e480feec34a0d6b0275663f1d504507c800719e436d97"} Sep 06 19:49:56.104572 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:56.104522 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" podStartSLOduration=2.104506124 podStartE2EDuration="2.104506124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:49:56.103211299 +0000 UTC m=+1338.837816030" watchObservedRunningTime="2026-09-06 19:49:56.104506124 +0000 UTC m=+1338.839110854" Sep 06 19:49:57.096019 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:57.095987 2575 generic.go:356] "Generic (PLEG): container finished" podID="132929b4-dc54-4e2f-bfaf-7c40acf200e4" containerID="27c4efbe59f36e07cb5a1afc3a75df73b04d4d1465e9c40e3be6086ca2f90e11" exitCode=0 Sep 06 19:49:57.096414 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:57.096071 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" event={"ID":"132929b4-dc54-4e2f-bfaf-7c40acf200e4","Type":"ContainerDied","Data":"27c4efbe59f36e07cb5a1afc3a75df73b04d4d1465e9c40e3be6086ca2f90e11"} Sep 06 19:49:58.237701 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:58.237674 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:58.366159 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:58.366064 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/132929b4-dc54-4e2f-bfaf-7c40acf200e4-maas-api-service-ca\") pod \"132929b4-dc54-4e2f-bfaf-7c40acf200e4\" (UID: \"132929b4-dc54-4e2f-bfaf-7c40acf200e4\") " Sep 06 19:49:58.366430 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:58.366406 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132929b4-dc54-4e2f-bfaf-7c40acf200e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "132929b4-dc54-4e2f-bfaf-7c40acf200e4" (UID: "132929b4-dc54-4e2f-bfaf-7c40acf200e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:49:58.467212 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:58.467177 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/132929b4-dc54-4e2f-bfaf-7c40acf200e4-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:49:59.108271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:59.108241 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" Sep 06 19:49:59.108447 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:59.108241 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk" event={"ID":"132929b4-dc54-4e2f-bfaf-7c40acf200e4","Type":"ContainerDied","Data":"2436205df948adf40b6e480feec34a0d6b0275663f1d504507c800719e436d97"} Sep 06 19:49:59.108447 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:49:59.108366 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2436205df948adf40b6e480feec34a0d6b0275663f1d504507c800719e436d97" Sep 06 19:50:10.501913 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.501876 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr"] Sep 06 19:50:10.502512 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.502145 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" podUID="dd0a7d5b-f327-496f-8113-a1482be79a82" containerName="payload-pre-processing" containerID="cri-o://2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63" gracePeriod=30 Sep 06 19:50:10.657891 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.657855 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk"] Sep 06 19:50:10.660181 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.660157 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0936e642-sknvk"] Sep 06 19:50:10.773148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.773116 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:50:10.889320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.889275 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjjkz\" (UniqueName: \"kubernetes.io/projected/dd0a7d5b-f327-496f-8113-a1482be79a82-kube-api-access-cjjkz\") pod \"dd0a7d5b-f327-496f-8113-a1482be79a82\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " Sep 06 19:50:10.889320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.889327 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0a7d5b-f327-496f-8113-a1482be79a82-plugins-config-volume\") pod \"dd0a7d5b-f327-496f-8113-a1482be79a82\" (UID: \"dd0a7d5b-f327-496f-8113-a1482be79a82\") " Sep 06 19:50:10.889791 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.889727 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd0a7d5b-f327-496f-8113-a1482be79a82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd0a7d5b-f327-496f-8113-a1482be79a82" (UID: "dd0a7d5b-f327-496f-8113-a1482be79a82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:50:10.891580 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.891557 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd0a7d5b-f327-496f-8113-a1482be79a82-kube-api-access-cjjkz" (OuterVolumeSpecName: "kube-api-access-cjjkz") pod "dd0a7d5b-f327-496f-8113-a1482be79a82" (UID: "dd0a7d5b-f327-496f-8113-a1482be79a82"). InnerVolumeSpecName "kube-api-access-cjjkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:50:10.990829 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.990795 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjjkz\" (UniqueName: \"kubernetes.io/projected/dd0a7d5b-f327-496f-8113-a1482be79a82-kube-api-access-cjjkz\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:50:10.990829 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:10.990827 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0a7d5b-f327-496f-8113-a1482be79a82-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:50:11.185205 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.185169 2575 generic.go:356] "Generic (PLEG): container finished" podID="dd0a7d5b-f327-496f-8113-a1482be79a82" containerID="2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63" exitCode=0 Sep 06 19:50:11.185205 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.185205 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" Sep 06 19:50:11.185426 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.185248 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" event={"ID":"dd0a7d5b-f327-496f-8113-a1482be79a82","Type":"ContainerDied","Data":"2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63"} Sep 06 19:50:11.185426 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.185308 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr" event={"ID":"dd0a7d5b-f327-496f-8113-a1482be79a82","Type":"ContainerDied","Data":"562f448ee8019b1e27aeed8794e6a24af92a30900d8adf149720e879348ae794"} Sep 06 19:50:11.185426 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.185334 2575 scope.go:117] "RemoveContainer" containerID="2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63" Sep 06 19:50:11.195602 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.195573 2575 scope.go:117] "RemoveContainer" containerID="2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63" Sep 06 19:50:11.195916 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:50:11.195898 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63\": container with ID starting with 2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63 not found: ID does not exist" containerID="2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63" Sep 06 19:50:11.195979 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.195933 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63"} err="failed to get container status \"2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63\": rpc error: code = NotFound desc = could not find container \"2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63\": container with ID starting with 2052d982dbc978785c77cc4ec12e5c3d7a89292d7a9ac071169dffd9b5f6dd63 not found: ID does not exist" Sep 06 19:50:11.205702 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.205677 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr"] Sep 06 19:50:11.209606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.209584 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0936e642-678b95c4bb-dx5gr"] Sep 06 19:50:11.762055 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.762022 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="132929b4-dc54-4e2f-bfaf-7c40acf200e4" path="/var/lib/kubelet/pods/132929b4-dc54-4e2f-bfaf-7c40acf200e4/volumes" Sep 06 19:50:11.762446 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:50:11.762372 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd0a7d5b-f327-496f-8113-a1482be79a82" path="/var/lib/kubelet/pods/dd0a7d5b-f327-496f-8113-a1482be79a82/volumes" Sep 06 19:51:37.902427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:51:37.902353 2575 scope.go:117] "RemoveContainer" containerID="94f9e51b7373184aa9f42758dc35d0489cfcaf790ac7a55cb737535e5343b21d" Sep 06 19:51:37.911598 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:51:37.911568 2575 scope.go:117] "RemoveContainer" containerID="4aa87c3646c366b7a14ceb52861ed6d3f23fd571d377636567941421147c2e0e" Sep 06 19:51:37.919863 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:51:37.919842 2575 scope.go:117] "RemoveContainer" containerID="47528678be9f26f8f8bfcf15a7b799cb6b7f236ec5d64747c91216a80605f4d9" Sep 06 19:52:16.417116 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417083 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls"] Sep 06 19:52:16.417536 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417502 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132929b4-dc54-4e2f-bfaf-7c40acf200e4" containerName="revoke-keys" Sep 06 19:52:16.417536 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417513 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="132929b4-dc54-4e2f-bfaf-7c40acf200e4" containerName="revoke-keys" Sep 06 19:52:16.417606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417537 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd0a7d5b-f327-496f-8113-a1482be79a82" containerName="payload-pre-processing" Sep 06 19:52:16.417606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417544 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd0a7d5b-f327-496f-8113-a1482be79a82" containerName="payload-pre-processing" Sep 06 19:52:16.417606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417595 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd0a7d5b-f327-496f-8113-a1482be79a82" containerName="payload-pre-processing" Sep 06 19:52:16.417606 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.417602 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="132929b4-dc54-4e2f-bfaf-7c40acf200e4" containerName="revoke-keys" Sep 06 19:52:16.420673 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.420657 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.423282 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.423257 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-cf16ea-dockercfg-7nzz4\"" Sep 06 19:52:16.423563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.423542 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-cf16ea\"" Sep 06 19:52:16.431118 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.431094 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls"] Sep 06 19:52:16.439536 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.439506 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbl6z\" (UniqueName: \"kubernetes.io/projected/b8e06171-1062-4369-8110-288228ee6eaa-kube-api-access-vbl6z\") pod \"payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.439663 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.439584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e06171-1062-4369-8110-288228ee6eaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.540349 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.540312 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbl6z\" (UniqueName: \"kubernetes.io/projected/b8e06171-1062-4369-8110-288228ee6eaa-kube-api-access-vbl6z\") pod \"payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.540525 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.540366 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e06171-1062-4369-8110-288228ee6eaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.541095 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.541076 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e06171-1062-4369-8110-288228ee6eaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.547966 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.547937 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbl6z\" (UniqueName: \"kubernetes.io/projected/b8e06171-1062-4369-8110-288228ee6eaa-kube-api-access-vbl6z\") pod \"payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:16.732718 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:16.732623 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:17.004193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:17.004168 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls"] Sep 06 19:52:17.005865 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:52:17.005824 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e06171_1062_4369_8110_288228ee6eaa.slice/crio-9aa2ac5e5cf185c61de12f553f22ff1ed0a99625324f8d8be008c2baec17c699 WatchSource:0}: Error finding container 9aa2ac5e5cf185c61de12f553f22ff1ed0a99625324f8d8be008c2baec17c699: Status 404 returned error can't find the container with id 9aa2ac5e5cf185c61de12f553f22ff1ed0a99625324f8d8be008c2baec17c699 Sep 06 19:52:17.917476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:17.916914 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" event={"ID":"b8e06171-1062-4369-8110-288228ee6eaa","Type":"ContainerStarted","Data":"35890775abcb2b8d16e44924ea2218ed5253e047006f0f16196440e0cb5d8193"} Sep 06 19:52:17.917476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:17.916990 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" event={"ID":"b8e06171-1062-4369-8110-288228ee6eaa","Type":"ContainerStarted","Data":"9aa2ac5e5cf185c61de12f553f22ff1ed0a99625324f8d8be008c2baec17c699"} Sep 06 19:52:17.917476 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:17.917198 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:17.934222 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:17.934170 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" podStartSLOduration=1.934153492 podStartE2EDuration="1.934153492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:52:17.930809688 +0000 UTC m=+1480.665414417" watchObservedRunningTime="2026-09-06 19:52:17.934153492 +0000 UTC m=+1480.668758223" Sep 06 19:52:28.924961 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:28.924924 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:52:37.802140 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:37.802114 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:52:37.805599 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:37.805578 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:52:37.948654 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:37.948621 2575 scope.go:117] "RemoveContainer" containerID="abd637c13469d5996e02bdfaac4f2d1232a5f34999634b95c1fa3cfeb1b74320" Sep 06 19:52:37.957441 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:37.957421 2575 scope.go:117] "RemoveContainer" containerID="550b11cac4005694de047accafcf927e60d321cd8c137f5171d86db5c433e1c8" Sep 06 19:52:45.652368 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.652335 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd"] Sep 06 19:52:45.657381 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.657362 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.659968 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.659946 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d50bae\"" Sep 06 19:52:45.660090 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.659968 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d50bae-dockercfg-mdn24\"" Sep 06 19:52:45.664739 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.664709 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd"] Sep 06 19:52:45.683385 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.683352 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr"] Sep 06 19:52:45.687581 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.687559 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.695929 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.695904 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr"] Sep 06 19:52:45.835149 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.835105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ceeb20-0420-4997-9144-0f78a7ca6aed-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.835149 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.835149 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk9bt\" (UniqueName: \"kubernetes.io/projected/cd83199c-19e9-4f25-bc02-fdd1ae224258-kube-api-access-vk9bt\") pod \"payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.835402 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.835263 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd83199c-19e9-4f25-bc02-fdd1ae224258-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.835402 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.835320 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrm2q\" (UniqueName: \"kubernetes.io/projected/08ceeb20-0420-4997-9144-0f78a7ca6aed-kube-api-access-qrm2q\") pod \"payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.936618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.936570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ceeb20-0420-4997-9144-0f78a7ca6aed-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.936618 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.936620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk9bt\" (UniqueName: \"kubernetes.io/projected/cd83199c-19e9-4f25-bc02-fdd1ae224258-kube-api-access-vk9bt\") pod \"payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.936951 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.936689 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd83199c-19e9-4f25-bc02-fdd1ae224258-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.936951 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.936725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrm2q\" (UniqueName: \"kubernetes.io/projected/08ceeb20-0420-4997-9144-0f78a7ca6aed-kube-api-access-qrm2q\") pod \"payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.937409 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.937389 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ceeb20-0420-4997-9144-0f78a7ca6aed-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.937454 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.937395 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd83199c-19e9-4f25-bc02-fdd1ae224258-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.946712 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.946688 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk9bt\" (UniqueName: \"kubernetes.io/projected/cd83199c-19e9-4f25-bc02-fdd1ae224258-kube-api-access-vk9bt\") pod \"payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:45.946929 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.946908 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrm2q\" (UniqueName: \"kubernetes.io/projected/08ceeb20-0420-4997-9144-0f78a7ca6aed-kube-api-access-qrm2q\") pod \"payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.970548 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.970520 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:45.999367 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:45.999332 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:46.249809 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:46.249764 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd"] Sep 06 19:52:46.252055 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:52:46.252026 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ceeb20_0420_4997_9144_0f78a7ca6aed.slice/crio-d675924c57e8af7cd9f9932c92f1d178db54dbc8278e0c639bbe64c8ddf779a8 WatchSource:0}: Error finding container d675924c57e8af7cd9f9932c92f1d178db54dbc8278e0c639bbe64c8ddf779a8: Status 404 returned error can't find the container with id d675924c57e8af7cd9f9932c92f1d178db54dbc8278e0c639bbe64c8ddf779a8 Sep 06 19:52:46.292677 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:46.292643 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr"] Sep 06 19:52:46.297872 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:52:46.297846 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd83199c_19e9_4f25_bc02_fdd1ae224258.slice/crio-b2f3df880bec4a9d8e5a06a32979febac2f7e1fe36e5cfcae013e658f1cc6444 WatchSource:0}: Error finding container b2f3df880bec4a9d8e5a06a32979febac2f7e1fe36e5cfcae013e658f1cc6444: Status 404 returned error can't find the container with id b2f3df880bec4a9d8e5a06a32979febac2f7e1fe36e5cfcae013e658f1cc6444 Sep 06 19:52:47.082256 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.082216 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" event={"ID":"08ceeb20-0420-4997-9144-0f78a7ca6aed","Type":"ContainerStarted","Data":"48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717"} Sep 06 19:52:47.082728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.082274 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" event={"ID":"08ceeb20-0420-4997-9144-0f78a7ca6aed","Type":"ContainerStarted","Data":"d675924c57e8af7cd9f9932c92f1d178db54dbc8278e0c639bbe64c8ddf779a8"} Sep 06 19:52:47.082728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.082295 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:47.083882 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.083852 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" event={"ID":"cd83199c-19e9-4f25-bc02-fdd1ae224258","Type":"ContainerStarted","Data":"aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6"} Sep 06 19:52:47.084029 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.083900 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" event={"ID":"cd83199c-19e9-4f25-bc02-fdd1ae224258","Type":"ContainerStarted","Data":"b2f3df880bec4a9d8e5a06a32979febac2f7e1fe36e5cfcae013e658f1cc6444"} Sep 06 19:52:47.084029 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.083923 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:52:47.096451 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.096396 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" podStartSLOduration=2.096379834 podStartE2EDuration="2.096379834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:52:47.096126423 +0000 UTC m=+1509.830731156" watchObservedRunningTime="2026-09-06 19:52:47.096379834 +0000 UTC m=+1509.830984566" Sep 06 19:52:47.114028 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:47.113980 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" podStartSLOduration=2.113966645 podStartE2EDuration="2.113966645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:52:47.1112028 +0000 UTC m=+1509.845807537" watchObservedRunningTime="2026-09-06 19:52:47.113966645 +0000 UTC m=+1509.848571376" Sep 06 19:52:58.092584 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:58.092550 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:52:58.093016 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:52:58.092620 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:53:11.546823 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.546708 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455"] Sep 06 19:53:11.550438 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.550418 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:11.553118 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.553090 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:53:11.553265 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.553102 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:53:11.556103 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.556074 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455"] Sep 06 19:53:11.682884 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.682849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e957515f-6fae-469a-8f94-d1b82dec1d6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cf16ea-dk455\" (UID: \"e957515f-6fae-469a-8f94-d1b82dec1d6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:11.783570 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.783532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e957515f-6fae-469a-8f94-d1b82dec1d6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cf16ea-dk455\" (UID: \"e957515f-6fae-469a-8f94-d1b82dec1d6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:11.784175 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.784153 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e957515f-6fae-469a-8f94-d1b82dec1d6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cf16ea-dk455\" (UID: \"e957515f-6fae-469a-8f94-d1b82dec1d6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:11.865153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:11.865058 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:12.145018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:12.143297 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455"] Sep 06 19:53:12.226893 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:12.226861 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" event={"ID":"e957515f-6fae-469a-8f94-d1b82dec1d6b","Type":"ContainerStarted","Data":"7c60f72e36b8b871c9d6c32a5f08292e4e22ab6139f9cc4e504dcf32b209508f"} Sep 06 19:53:13.234205 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:13.234170 2575 generic.go:356] "Generic (PLEG): container finished" podID="e957515f-6fae-469a-8f94-d1b82dec1d6b" containerID="b1eec8c1bfe448395d41e56f1fe85d4861834542781bbdb393f45aa97f1b883d" exitCode=0 Sep 06 19:53:13.234600 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:13.234263 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" event={"ID":"e957515f-6fae-469a-8f94-d1b82dec1d6b","Type":"ContainerDied","Data":"b1eec8c1bfe448395d41e56f1fe85d4861834542781bbdb393f45aa97f1b883d"} Sep 06 19:53:14.383479 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:14.383453 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:14.507461 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:14.507375 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e957515f-6fae-469a-8f94-d1b82dec1d6b-maas-api-service-ca\") pod \"e957515f-6fae-469a-8f94-d1b82dec1d6b\" (UID: \"e957515f-6fae-469a-8f94-d1b82dec1d6b\") " Sep 06 19:53:14.507735 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:14.507711 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e957515f-6fae-469a-8f94-d1b82dec1d6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e957515f-6fae-469a-8f94-d1b82dec1d6b" (UID: "e957515f-6fae-469a-8f94-d1b82dec1d6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:53:14.608957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:14.608918 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e957515f-6fae-469a-8f94-d1b82dec1d6b-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:15.247717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:15.247680 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" event={"ID":"e957515f-6fae-469a-8f94-d1b82dec1d6b","Type":"ContainerDied","Data":"7c60f72e36b8b871c9d6c32a5f08292e4e22ab6139f9cc4e504dcf32b209508f"} Sep 06 19:53:15.247717 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:15.247703 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455" Sep 06 19:53:15.247961 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:15.247792 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c60f72e36b8b871c9d6c32a5f08292e4e22ab6139f9cc4e504dcf32b209508f" Sep 06 19:53:27.102935 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.102896 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls"] Sep 06 19:53:27.104168 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.104134 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" podUID="b8e06171-1062-4369-8110-288228ee6eaa" containerName="payload-pre-processing" containerID="cri-o://35890775abcb2b8d16e44924ea2218ed5253e047006f0f16196440e0cb5d8193" gracePeriod=30 Sep 06 19:53:27.233247 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.233213 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455"] Sep 06 19:53:27.237617 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.237591 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf16ea-dk455"] Sep 06 19:53:27.328651 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.328613 2575 generic.go:356] "Generic (PLEG): container finished" podID="b8e06171-1062-4369-8110-288228ee6eaa" containerID="35890775abcb2b8d16e44924ea2218ed5253e047006f0f16196440e0cb5d8193" exitCode=0 Sep 06 19:53:27.328865 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.328773 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" event={"ID":"b8e06171-1062-4369-8110-288228ee6eaa","Type":"ContainerDied","Data":"35890775abcb2b8d16e44924ea2218ed5253e047006f0f16196440e0cb5d8193"} Sep 06 19:53:27.376822 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.376799 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:53:27.541599 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.541567 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e06171-1062-4369-8110-288228ee6eaa-plugins-config-volume\") pod \"b8e06171-1062-4369-8110-288228ee6eaa\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " Sep 06 19:53:27.541793 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.541623 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbl6z\" (UniqueName: \"kubernetes.io/projected/b8e06171-1062-4369-8110-288228ee6eaa-kube-api-access-vbl6z\") pod \"b8e06171-1062-4369-8110-288228ee6eaa\" (UID: \"b8e06171-1062-4369-8110-288228ee6eaa\") " Sep 06 19:53:27.541976 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.541950 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8e06171-1062-4369-8110-288228ee6eaa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8e06171-1062-4369-8110-288228ee6eaa" (UID: "b8e06171-1062-4369-8110-288228ee6eaa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:53:27.543946 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.543923 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e06171-1062-4369-8110-288228ee6eaa-kube-api-access-vbl6z" (OuterVolumeSpecName: "kube-api-access-vbl6z") pod "b8e06171-1062-4369-8110-288228ee6eaa" (UID: "b8e06171-1062-4369-8110-288228ee6eaa"). InnerVolumeSpecName "kube-api-access-vbl6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:53:27.642963 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.642872 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e06171-1062-4369-8110-288228ee6eaa-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:27.642963 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.642904 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbl6z\" (UniqueName: \"kubernetes.io/projected/b8e06171-1062-4369-8110-288228ee6eaa-kube-api-access-vbl6z\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:27.763381 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:27.763350 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e957515f-6fae-469a-8f94-d1b82dec1d6b" path="/var/lib/kubelet/pods/e957515f-6fae-469a-8f94-d1b82dec1d6b/volumes" Sep 06 19:53:28.335807 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:28.335772 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" event={"ID":"b8e06171-1062-4369-8110-288228ee6eaa","Type":"ContainerDied","Data":"9aa2ac5e5cf185c61de12f553f22ff1ed0a99625324f8d8be008c2baec17c699"} Sep 06 19:53:28.335807 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:28.335790 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls" Sep 06 19:53:28.336314 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:28.335858 2575 scope.go:117] "RemoveContainer" containerID="35890775abcb2b8d16e44924ea2218ed5253e047006f0f16196440e0cb5d8193" Sep 06 19:53:28.353159 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:28.353136 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls"] Sep 06 19:53:28.355620 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:28.355598 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf16ea-6ffb875678-4h9ls"] Sep 06 19:53:29.763138 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:29.763094 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8e06171-1062-4369-8110-288228ee6eaa" path="/var/lib/kubelet/pods/b8e06171-1062-4369-8110-288228ee6eaa/volumes" Sep 06 19:53:33.866007 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.865969 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t"] Sep 06 19:53:33.866409 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.866379 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8e06171-1062-4369-8110-288228ee6eaa" containerName="payload-pre-processing" Sep 06 19:53:33.866409 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.866390 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8e06171-1062-4369-8110-288228ee6eaa" containerName="payload-pre-processing" Sep 06 19:53:33.866409 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.866408 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e957515f-6fae-469a-8f94-d1b82dec1d6b" containerName="revoke-keys" Sep 06 19:53:33.866525 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.866414 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e957515f-6fae-469a-8f94-d1b82dec1d6b" containerName="revoke-keys" Sep 06 19:53:33.866525 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.866475 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8e06171-1062-4369-8110-288228ee6eaa" containerName="payload-pre-processing" Sep 06 19:53:33.866525 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.866485 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e957515f-6fae-469a-8f94-d1b82dec1d6b" containerName="revoke-keys" Sep 06 19:53:33.870959 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.870942 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:33.874245 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.874222 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:53:33.874563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.874223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:53:33.876814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.876789 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t"] Sep 06 19:53:33.908032 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:33.907989 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7753b55-bb56-4703-b7c1-a28c20bb57f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d50bae-qth6t\" (UID: \"b7753b55-bb56-4703-b7c1-a28c20bb57f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:34.008942 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:34.008905 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7753b55-bb56-4703-b7c1-a28c20bb57f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d50bae-qth6t\" (UID: \"b7753b55-bb56-4703-b7c1-a28c20bb57f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:34.009493 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:34.009474 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7753b55-bb56-4703-b7c1-a28c20bb57f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d50bae-qth6t\" (UID: \"b7753b55-bb56-4703-b7c1-a28c20bb57f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:34.185742 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:34.185709 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:34.466347 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:34.466318 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t"] Sep 06 19:53:34.467895 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:53:34.467871 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7753b55_bb56_4703_b7c1_a28c20bb57f6.slice/crio-fb161eddfaa942e421666263ce976704431854bf47b7c690d5cbc1215d187587 WatchSource:0}: Error finding container fb161eddfaa942e421666263ce976704431854bf47b7c690d5cbc1215d187587: Status 404 returned error can't find the container with id fb161eddfaa942e421666263ce976704431854bf47b7c690d5cbc1215d187587 Sep 06 19:53:35.383728 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:35.383697 2575 generic.go:356] "Generic (PLEG): container finished" podID="b7753b55-bb56-4703-b7c1-a28c20bb57f6" containerID="e0bc81e0355d36fd9e0ffd252ecdbcdd74fcfa96f341f6e7316558900a3e5bae" exitCode=0 Sep 06 19:53:35.384111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:35.383788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" event={"ID":"b7753b55-bb56-4703-b7c1-a28c20bb57f6","Type":"ContainerDied","Data":"e0bc81e0355d36fd9e0ffd252ecdbcdd74fcfa96f341f6e7316558900a3e5bae"} Sep 06 19:53:35.384111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:35.383850 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" event={"ID":"b7753b55-bb56-4703-b7c1-a28c20bb57f6","Type":"ContainerStarted","Data":"fb161eddfaa942e421666263ce976704431854bf47b7c690d5cbc1215d187587"} Sep 06 19:53:36.528225 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:36.528191 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:36.634769 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:36.634723 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7753b55-bb56-4703-b7c1-a28c20bb57f6-maas-api-service-ca\") pod \"b7753b55-bb56-4703-b7c1-a28c20bb57f6\" (UID: \"b7753b55-bb56-4703-b7c1-a28c20bb57f6\") " Sep 06 19:53:36.635053 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:36.635031 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7753b55-bb56-4703-b7c1-a28c20bb57f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7753b55-bb56-4703-b7c1-a28c20bb57f6" (UID: "b7753b55-bb56-4703-b7c1-a28c20bb57f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:53:36.735837 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:36.735804 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7753b55-bb56-4703-b7c1-a28c20bb57f6-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:37.396257 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:37.396224 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" Sep 06 19:53:37.396443 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:37.396265 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t" event={"ID":"b7753b55-bb56-4703-b7c1-a28c20bb57f6","Type":"ContainerDied","Data":"fb161eddfaa942e421666263ce976704431854bf47b7c690d5cbc1215d187587"} Sep 06 19:53:37.396443 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:37.396311 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb161eddfaa942e421666263ce976704431854bf47b7c690d5cbc1215d187587" Sep 06 19:53:49.360173 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.360133 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr"] Sep 06 19:53:49.361199 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.361143 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" podUID="cd83199c-19e9-4f25-bc02-fdd1ae224258" containerName="payload-processing" containerID="cri-o://aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6" gracePeriod=30 Sep 06 19:53:49.378335 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.378301 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd"] Sep 06 19:53:49.378566 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.378520 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" podUID="08ceeb20-0420-4997-9144-0f78a7ca6aed" containerName="payload-pre-processing" containerID="cri-o://48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717" gracePeriod=30 Sep 06 19:53:49.508018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.507965 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t"] Sep 06 19:53:49.510536 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.510495 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d50bae-qth6t"] Sep 06 19:53:49.672486 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.672460 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:53:49.675456 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.675438 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:53:49.762339 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.762306 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7753b55-bb56-4703-b7c1-a28c20bb57f6" path="/var/lib/kubelet/pods/b7753b55-bb56-4703-b7c1-a28c20bb57f6/volumes" Sep 06 19:53:49.764043 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.764019 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd83199c-19e9-4f25-bc02-fdd1ae224258-plugins-config-volume\") pod \"cd83199c-19e9-4f25-bc02-fdd1ae224258\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " Sep 06 19:53:49.764148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.764096 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk9bt\" (UniqueName: \"kubernetes.io/projected/cd83199c-19e9-4f25-bc02-fdd1ae224258-kube-api-access-vk9bt\") pod \"cd83199c-19e9-4f25-bc02-fdd1ae224258\" (UID: \"cd83199c-19e9-4f25-bc02-fdd1ae224258\") " Sep 06 19:53:49.764148 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.764143 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ceeb20-0420-4997-9144-0f78a7ca6aed-plugins-config-volume\") pod \"08ceeb20-0420-4997-9144-0f78a7ca6aed\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " Sep 06 19:53:49.764275 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.764194 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrm2q\" (UniqueName: \"kubernetes.io/projected/08ceeb20-0420-4997-9144-0f78a7ca6aed-kube-api-access-qrm2q\") pod \"08ceeb20-0420-4997-9144-0f78a7ca6aed\" (UID: \"08ceeb20-0420-4997-9144-0f78a7ca6aed\") " Sep 06 19:53:49.764494 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.764465 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd83199c-19e9-4f25-bc02-fdd1ae224258-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd83199c-19e9-4f25-bc02-fdd1ae224258" (UID: "cd83199c-19e9-4f25-bc02-fdd1ae224258"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:53:49.764588 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.764521 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ceeb20-0420-4997-9144-0f78a7ca6aed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08ceeb20-0420-4997-9144-0f78a7ca6aed" (UID: "08ceeb20-0420-4997-9144-0f78a7ca6aed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:53:49.766404 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.766384 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd83199c-19e9-4f25-bc02-fdd1ae224258-kube-api-access-vk9bt" (OuterVolumeSpecName: "kube-api-access-vk9bt") pod "cd83199c-19e9-4f25-bc02-fdd1ae224258" (UID: "cd83199c-19e9-4f25-bc02-fdd1ae224258"). InnerVolumeSpecName "kube-api-access-vk9bt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:53:49.766486 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.766446 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08ceeb20-0420-4997-9144-0f78a7ca6aed-kube-api-access-qrm2q" (OuterVolumeSpecName: "kube-api-access-qrm2q") pod "08ceeb20-0420-4997-9144-0f78a7ca6aed" (UID: "08ceeb20-0420-4997-9144-0f78a7ca6aed"). InnerVolumeSpecName "kube-api-access-qrm2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:53:49.865824 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.865787 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ceeb20-0420-4997-9144-0f78a7ca6aed-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:49.865824 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.865824 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrm2q\" (UniqueName: \"kubernetes.io/projected/08ceeb20-0420-4997-9144-0f78a7ca6aed-kube-api-access-qrm2q\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:49.866023 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.865839 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd83199c-19e9-4f25-bc02-fdd1ae224258-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:49.866023 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:49.865852 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk9bt\" (UniqueName: \"kubernetes.io/projected/cd83199c-19e9-4f25-bc02-fdd1ae224258-kube-api-access-vk9bt\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:50.472350 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.472319 2575 generic.go:356] "Generic (PLEG): container finished" podID="cd83199c-19e9-4f25-bc02-fdd1ae224258" containerID="aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6" exitCode=0 Sep 06 19:53:50.472784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.472360 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" Sep 06 19:53:50.472784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.472401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" event={"ID":"cd83199c-19e9-4f25-bc02-fdd1ae224258","Type":"ContainerDied","Data":"aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6"} Sep 06 19:53:50.472784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.472456 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr" event={"ID":"cd83199c-19e9-4f25-bc02-fdd1ae224258","Type":"ContainerDied","Data":"b2f3df880bec4a9d8e5a06a32979febac2f7e1fe36e5cfcae013e658f1cc6444"} Sep 06 19:53:50.472784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.472483 2575 scope.go:117] "RemoveContainer" containerID="aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6" Sep 06 19:53:50.474008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.473815 2575 generic.go:356] "Generic (PLEG): container finished" podID="08ceeb20-0420-4997-9144-0f78a7ca6aed" containerID="48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717" exitCode=0 Sep 06 19:53:50.474008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.473863 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" event={"ID":"08ceeb20-0420-4997-9144-0f78a7ca6aed","Type":"ContainerDied","Data":"48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717"} Sep 06 19:53:50.474008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.473886 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" Sep 06 19:53:50.474008 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.473901 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd" event={"ID":"08ceeb20-0420-4997-9144-0f78a7ca6aed","Type":"ContainerDied","Data":"d675924c57e8af7cd9f9932c92f1d178db54dbc8278e0c639bbe64c8ddf779a8"} Sep 06 19:53:50.483993 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.483972 2575 scope.go:117] "RemoveContainer" containerID="aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6" Sep 06 19:53:50.484268 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:53:50.484246 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6\": container with ID starting with aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6 not found: ID does not exist" containerID="aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6" Sep 06 19:53:50.484399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.484276 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6"} err="failed to get container status \"aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6\": rpc error: code = NotFound desc = could not find container \"aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6\": container with ID starting with aca62e4b0680bf8dc507410e4fc52f75cf4be9842e8cfdf6fcc2724477a18ac6 not found: ID does not exist" Sep 06 19:53:50.484399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.484301 2575 scope.go:117] "RemoveContainer" containerID="48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717" Sep 06 19:53:50.492809 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.492786 2575 scope.go:117] "RemoveContainer" containerID="48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717" Sep 06 19:53:50.493081 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:53:50.493061 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717\": container with ID starting with 48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717 not found: ID does not exist" containerID="48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717" Sep 06 19:53:50.493138 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.493087 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717"} err="failed to get container status \"48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717\": rpc error: code = NotFound desc = could not find container \"48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717\": container with ID starting with 48ddb14b27f4de1245c5c6c0c75499f08bda0b7d44b51df9d3e6765d175fe717 not found: ID does not exist" Sep 06 19:53:50.498651 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.498631 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd"] Sep 06 19:53:50.502277 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.502257 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d50bae-549cb68fd-snvfd"] Sep 06 19:53:50.512094 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.512074 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr"] Sep 06 19:53:50.519352 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:50.519333 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d50bae-5d58979d9f-7hzkr"] Sep 06 19:53:51.762877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:51.762846 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08ceeb20-0420-4997-9144-0f78a7ca6aed" path="/var/lib/kubelet/pods/08ceeb20-0420-4997-9144-0f78a7ca6aed/volumes" Sep 06 19:53:51.763249 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:51.763199 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd83199c-19e9-4f25-bc02-fdd1ae224258" path="/var/lib/kubelet/pods/cd83199c-19e9-4f25-bc02-fdd1ae224258/volumes" Sep 06 19:53:53.069439 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.069407 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p"] Sep 06 19:53:53.070055 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070036 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7753b55-bb56-4703-b7c1-a28c20bb57f6" containerName="revoke-keys" Sep 06 19:53:53.070153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070058 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7753b55-bb56-4703-b7c1-a28c20bb57f6" containerName="revoke-keys" Sep 06 19:53:53.070153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070080 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08ceeb20-0420-4997-9144-0f78a7ca6aed" containerName="payload-pre-processing" Sep 06 19:53:53.070153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070088 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ceeb20-0420-4997-9144-0f78a7ca6aed" containerName="payload-pre-processing" Sep 06 19:53:53.070153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070110 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd83199c-19e9-4f25-bc02-fdd1ae224258" containerName="payload-processing" Sep 06 19:53:53.070153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070119 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd83199c-19e9-4f25-bc02-fdd1ae224258" containerName="payload-processing" Sep 06 19:53:53.070403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070232 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd83199c-19e9-4f25-bc02-fdd1ae224258" containerName="payload-processing" Sep 06 19:53:53.070403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070245 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7753b55-bb56-4703-b7c1-a28c20bb57f6" containerName="revoke-keys" Sep 06 19:53:53.070403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.070256 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="08ceeb20-0420-4997-9144-0f78a7ca6aed" containerName="payload-pre-processing" Sep 06 19:53:53.074998 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.074978 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.077561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.077541 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-a30d6c-openshift-default-dockercfg-z6h88\"" Sep 06 19:53:53.086056 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.086029 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p"] Sep 06 19:53:53.207738 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.207701 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-envoy\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.207946 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.207822 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pxhb\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-kube-api-access-2pxhb\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.207946 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.207845 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-certs\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.207946 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.207870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-socket\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.208060 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.207976 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-token\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.208060 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.208012 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-credential-socket\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.208060 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.208038 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-data\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.208158 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.208062 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-podinfo\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.208158 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.208084 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b404a2a-e6f3-4833-87ec-97c7031f428e-istiod-ca-cert\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309323 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309277 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-envoy\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309516 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309369 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pxhb\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-kube-api-access-2pxhb\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309516 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-certs\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309516 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-socket\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309516 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-token\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309516 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-credential-socket\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309841 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309800 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-data\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309937 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309852 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-certs\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309937 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-podinfo\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.309937 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309919 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b404a2a-e6f3-4833-87ec-97c7031f428e-istiod-ca-cert\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.310102 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.309952 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-socket\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.310102 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.310092 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-data\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.310281 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.310171 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-credential-socket\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.310530 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.310513 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b404a2a-e6f3-4833-87ec-97c7031f428e-istiod-ca-cert\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.311844 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.311825 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-envoy\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.312262 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.312244 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-podinfo\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.320738 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.320674 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-token\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.320955 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.320879 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pxhb\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-kube-api-access-2pxhb\") pod \"e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.388568 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.388534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:53.672734 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.672694 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p"] Sep 06 19:53:53.675674 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:53:53.675649 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b404a2a_e6f3_4833_87ec_97c7031f428e.slice/crio-02e712a952f99e6422fae4f2a0d7d61465f48a9dbcbec7a35b3e1b55235d4d1f WatchSource:0}: Error finding container 02e712a952f99e6422fae4f2a0d7d61465f48a9dbcbec7a35b3e1b55235d4d1f: Status 404 returned error can't find the container with id 02e712a952f99e6422fae4f2a0d7d61465f48a9dbcbec7a35b3e1b55235d4d1f Sep 06 19:53:53.677860 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.677827 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:53:53.677928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.677892 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:53:53.677928 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:53.677921 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:53:54.500982 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:54.500949 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" event={"ID":"7b404a2a-e6f3-4833-87ec-97c7031f428e","Type":"ContainerStarted","Data":"e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f"} Sep 06 19:53:54.501413 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:54.501000 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" event={"ID":"7b404a2a-e6f3-4833-87ec-97c7031f428e","Type":"ContainerStarted","Data":"02e712a952f99e6422fae4f2a0d7d61465f48a9dbcbec7a35b3e1b55235d4d1f"} Sep 06 19:53:54.521760 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:54.521699 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podStartSLOduration=1.5216849080000001 podStartE2EDuration="1.521684908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:53:54.518388888 +0000 UTC m=+1577.252993635" watchObservedRunningTime="2026-09-06 19:53:54.521684908 +0000 UTC m=+1577.256289639" Sep 06 19:53:55.389011 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:55.388976 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:55.394553 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:55.394527 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:55.506560 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:55.506529 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:55.507546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:55.507524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:53:56.183710 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.183675 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t"] Sep 06 19:53:56.187467 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.187443 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:56.190157 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.190133 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:53:56.190872 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.190849 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:53:56.193895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.193870 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t"] Sep 06 19:53:56.349141 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.349104 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4493b46e-cf02-44fe-95df-2f1cb3ba46d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t\" (UID: \"4493b46e-cf02-44fe-95df-2f1cb3ba46d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:56.450473 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.450399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4493b46e-cf02-44fe-95df-2f1cb3ba46d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t\" (UID: \"4493b46e-cf02-44fe-95df-2f1cb3ba46d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:56.451027 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.451005 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4493b46e-cf02-44fe-95df-2f1cb3ba46d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t\" (UID: \"4493b46e-cf02-44fe-95df-2f1cb3ba46d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:56.501018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.500983 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:56.774766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:56.774724 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t"] Sep 06 19:53:56.776274 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:53:56.776246 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4493b46e_cf02_44fe_95df_2f1cb3ba46d1.slice/crio-bda3d347bfd565e882f4de22f0d0d08ce880eb41297edb32cd86099b5d69f7c4 WatchSource:0}: Error finding container bda3d347bfd565e882f4de22f0d0d08ce880eb41297edb32cd86099b5d69f7c4: Status 404 returned error can't find the container with id bda3d347bfd565e882f4de22f0d0d08ce880eb41297edb32cd86099b5d69f7c4 Sep 06 19:53:57.520685 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:57.520653 2575 generic.go:356] "Generic (PLEG): container finished" podID="4493b46e-cf02-44fe-95df-2f1cb3ba46d1" containerID="e22ad390884c3e4d0d2d940026bdd678b07461ed2ee0edc9b7e43ccbfa62be24" exitCode=0 Sep 06 19:53:57.520831 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:57.520702 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" event={"ID":"4493b46e-cf02-44fe-95df-2f1cb3ba46d1","Type":"ContainerDied","Data":"e22ad390884c3e4d0d2d940026bdd678b07461ed2ee0edc9b7e43ccbfa62be24"} Sep 06 19:53:57.520831 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:57.520786 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" event={"ID":"4493b46e-cf02-44fe-95df-2f1cb3ba46d1","Type":"ContainerStarted","Data":"bda3d347bfd565e882f4de22f0d0d08ce880eb41297edb32cd86099b5d69f7c4"} Sep 06 19:53:58.305112 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.305076 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt"] Sep 06 19:53:58.308865 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.308843 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.311604 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.311588 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a30d6c\"" Sep 06 19:53:58.329343 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.329313 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt"] Sep 06 19:53:58.473672 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.473636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgljl\" (UniqueName: \"kubernetes.io/projected/938dfedc-198e-4c01-b083-1ad7918e638b-kube-api-access-sgljl\") pod \"payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.473852 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.473684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938dfedc-198e-4c01-b083-1ad7918e638b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.574798 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.574682 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgljl\" (UniqueName: \"kubernetes.io/projected/938dfedc-198e-4c01-b083-1ad7918e638b-kube-api-access-sgljl\") pod \"payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.574798 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.574725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938dfedc-198e-4c01-b083-1ad7918e638b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.575471 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.575440 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938dfedc-198e-4c01-b083-1ad7918e638b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.581795 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.581770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgljl\" (UniqueName: \"kubernetes.io/projected/938dfedc-198e-4c01-b083-1ad7918e638b-kube-api-access-sgljl\") pod \"payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.619042 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.619010 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:58.685624 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.685600 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:58.776298 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.776271 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4493b46e-cf02-44fe-95df-2f1cb3ba46d1-maas-api-service-ca\") pod \"4493b46e-cf02-44fe-95df-2f1cb3ba46d1\" (UID: \"4493b46e-cf02-44fe-95df-2f1cb3ba46d1\") " Sep 06 19:53:58.776624 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.776588 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4493b46e-cf02-44fe-95df-2f1cb3ba46d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4493b46e-cf02-44fe-95df-2f1cb3ba46d1" (UID: "4493b46e-cf02-44fe-95df-2f1cb3ba46d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:53:58.878528 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.878442 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4493b46e-cf02-44fe-95df-2f1cb3ba46d1-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:53:58.895341 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:58.895317 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt"] Sep 06 19:53:58.896371 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:53:58.896349 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938dfedc_198e_4c01_b083_1ad7918e638b.slice/crio-6d7128d88dbd7c3b38ae032a2a59fde514174309c9f5957eb64b8a1ca3cd0a86 WatchSource:0}: Error finding container 6d7128d88dbd7c3b38ae032a2a59fde514174309c9f5957eb64b8a1ca3cd0a86: Status 404 returned error can't find the container with id 6d7128d88dbd7c3b38ae032a2a59fde514174309c9f5957eb64b8a1ca3cd0a86 Sep 06 19:53:59.535085 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.535062 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" Sep 06 19:53:59.535521 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.535097 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t" event={"ID":"4493b46e-cf02-44fe-95df-2f1cb3ba46d1","Type":"ContainerDied","Data":"bda3d347bfd565e882f4de22f0d0d08ce880eb41297edb32cd86099b5d69f7c4"} Sep 06 19:53:59.535521 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.535149 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bda3d347bfd565e882f4de22f0d0d08ce880eb41297edb32cd86099b5d69f7c4" Sep 06 19:53:59.536890 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.536859 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" event={"ID":"938dfedc-198e-4c01-b083-1ad7918e638b","Type":"ContainerStarted","Data":"c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac"} Sep 06 19:53:59.536987 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.536903 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" event={"ID":"938dfedc-198e-4c01-b083-1ad7918e638b","Type":"ContainerStarted","Data":"6d7128d88dbd7c3b38ae032a2a59fde514174309c9f5957eb64b8a1ca3cd0a86"} Sep 06 19:53:59.537031 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.536995 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:53:59.552957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:53:59.552914 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" podStartSLOduration=1.552900714 podStartE2EDuration="1.552900714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:53:59.551096872 +0000 UTC m=+1582.285701604" watchObservedRunningTime="2026-09-06 19:53:59.552900714 +0000 UTC m=+1582.287505493" Sep 06 19:54:06.855272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:06.855235 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx"] Sep 06 19:54:06.855731 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:06.855579 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" podUID="458ec939-67e3-4d12-8800-21245aab56ad" containerName="payload-processing" containerID="cri-o://fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc" gracePeriod=30 Sep 06 19:54:06.860357 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:06.860323 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk"] Sep 06 19:54:06.860727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:06.860697 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" podUID="7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" containerName="payload-pre-processing" containerID="cri-o://8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db" gracePeriod=30 Sep 06 19:54:07.010507 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.010461 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t"] Sep 06 19:54:07.010866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.010849 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b12ed2-6lw5t"] Sep 06 19:54:07.180083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.180049 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:54:07.210784 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.208713 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:54:07.257573 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.257545 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkph8\" (UniqueName: \"kubernetes.io/projected/458ec939-67e3-4d12-8800-21245aab56ad-kube-api-access-xkph8\") pod \"458ec939-67e3-4d12-8800-21245aab56ad\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " Sep 06 19:54:07.257789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.257590 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-plugins-config-volume\") pod \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " Sep 06 19:54:07.257789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.257613 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phs7m\" (UniqueName: \"kubernetes.io/projected/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-kube-api-access-phs7m\") pod \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\" (UID: \"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44\") " Sep 06 19:54:07.257789 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.257640 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458ec939-67e3-4d12-8800-21245aab56ad-plugins-config-volume\") pod \"458ec939-67e3-4d12-8800-21245aab56ad\" (UID: \"458ec939-67e3-4d12-8800-21245aab56ad\") " Sep 06 19:54:07.258045 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.258019 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" (UID: "7ba5dff1-62a4-4fb0-9089-c55d30ec5f44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:54:07.258099 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.258072 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458ec939-67e3-4d12-8800-21245aab56ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "458ec939-67e3-4d12-8800-21245aab56ad" (UID: "458ec939-67e3-4d12-8800-21245aab56ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:54:07.260395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.260371 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458ec939-67e3-4d12-8800-21245aab56ad-kube-api-access-xkph8" (OuterVolumeSpecName: "kube-api-access-xkph8") pod "458ec939-67e3-4d12-8800-21245aab56ad" (UID: "458ec939-67e3-4d12-8800-21245aab56ad"). InnerVolumeSpecName "kube-api-access-xkph8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:54:07.260395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.260380 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-kube-api-access-phs7m" (OuterVolumeSpecName: "kube-api-access-phs7m") pod "7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" (UID: "7ba5dff1-62a4-4fb0-9089-c55d30ec5f44"). InnerVolumeSpecName "kube-api-access-phs7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:54:07.359057 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.359021 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:07.359057 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.359054 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phs7m\" (UniqueName: \"kubernetes.io/projected/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44-kube-api-access-phs7m\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:07.359057 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.359064 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458ec939-67e3-4d12-8800-21245aab56ad-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:07.359291 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.359072 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkph8\" (UniqueName: \"kubernetes.io/projected/458ec939-67e3-4d12-8800-21245aab56ad-kube-api-access-xkph8\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:07.587270 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.587236 2575 generic.go:356] "Generic (PLEG): container finished" podID="7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" containerID="8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db" exitCode=0 Sep 06 19:54:07.587470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.587279 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" Sep 06 19:54:07.587470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.587325 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" event={"ID":"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44","Type":"ContainerDied","Data":"8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db"} Sep 06 19:54:07.587470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.587417 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk" event={"ID":"7ba5dff1-62a4-4fb0-9089-c55d30ec5f44","Type":"ContainerDied","Data":"a8a0631bce3975b2b0b1cbeb2bbd8fef7381bac0c0c493bd278343db2dfcaffb"} Sep 06 19:54:07.587470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.587447 2575 scope.go:117] "RemoveContainer" containerID="8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db" Sep 06 19:54:07.588799 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.588776 2575 generic.go:356] "Generic (PLEG): container finished" podID="458ec939-67e3-4d12-8800-21245aab56ad" containerID="fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc" exitCode=0 Sep 06 19:54:07.588915 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.588806 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" event={"ID":"458ec939-67e3-4d12-8800-21245aab56ad","Type":"ContainerDied","Data":"fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc"} Sep 06 19:54:07.588915 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.588821 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" Sep 06 19:54:07.588915 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.588852 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx" event={"ID":"458ec939-67e3-4d12-8800-21245aab56ad","Type":"ContainerDied","Data":"b5f694ee53ae868b6b92e28a9ecbf25cd3eab13f0ee325861be4053491112dee"} Sep 06 19:54:07.617091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.616957 2575 scope.go:117] "RemoveContainer" containerID="8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db" Sep 06 19:54:07.617434 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:54:07.617398 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db\": container with ID starting with 8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db not found: ID does not exist" containerID="8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db" Sep 06 19:54:07.617541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.617449 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db"} err="failed to get container status \"8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db\": rpc error: code = NotFound desc = could not find container \"8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db\": container with ID starting with 8eaa8ff677dccc523e8b23892aed120dedff093f400bb6f28f47acdece7b33db not found: ID does not exist" Sep 06 19:54:07.617541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.617476 2575 scope.go:117] "RemoveContainer" containerID="fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc" Sep 06 19:54:07.628601 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.628582 2575 scope.go:117] "RemoveContainer" containerID="fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc" Sep 06 19:54:07.628948 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:54:07.628927 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc\": container with ID starting with fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc not found: ID does not exist" containerID="fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc" Sep 06 19:54:07.629013 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.628955 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc"} err="failed to get container status \"fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc\": rpc error: code = NotFound desc = could not find container \"fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc\": container with ID starting with fa0f0ce4db1c0f3b5e2a1e45b90e4163baf839abc0707889dafebe6c0a3faedc not found: ID does not exist" Sep 06 19:54:07.632892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.632868 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk"] Sep 06 19:54:07.636437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.636410 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b12ed2-77db4c5754-gk9bk"] Sep 06 19:54:07.651282 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.651257 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx"] Sep 06 19:54:07.656637 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.656613 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b12ed2-56d5c697c-h24mx"] Sep 06 19:54:07.762816 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.762785 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4493b46e-cf02-44fe-95df-2f1cb3ba46d1" path="/var/lib/kubelet/pods/4493b46e-cf02-44fe-95df-2f1cb3ba46d1/volumes" Sep 06 19:54:07.763150 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.763135 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="458ec939-67e3-4d12-8800-21245aab56ad" path="/var/lib/kubelet/pods/458ec939-67e3-4d12-8800-21245aab56ad/volumes" Sep 06 19:54:07.763447 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:07.763433 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" path="/var/lib/kubelet/pods/7ba5dff1-62a4-4fb0-9089-c55d30ec5f44/volumes" Sep 06 19:54:10.544866 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:10.544843 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:54:11.547933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:11.547894 2575 patch_prober.go:28] interesting pod/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 06 19:54:11.548320 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:11.547961 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 06 19:54:13.142537 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.142499 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd"] Sep 06 19:54:13.143077 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.142793 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" containerID="cri-o://993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399" gracePeriod=30 Sep 06 19:54:13.480649 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.480613 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn"] Sep 06 19:54:13.481059 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481045 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4493b46e-cf02-44fe-95df-2f1cb3ba46d1" containerName="revoke-keys" Sep 06 19:54:13.481108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481060 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4493b46e-cf02-44fe-95df-2f1cb3ba46d1" containerName="revoke-keys" Sep 06 19:54:13.481108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481080 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458ec939-67e3-4d12-8800-21245aab56ad" containerName="payload-processing" Sep 06 19:54:13.481108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481086 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="458ec939-67e3-4d12-8800-21245aab56ad" containerName="payload-processing" Sep 06 19:54:13.481108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481094 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" containerName="payload-pre-processing" Sep 06 19:54:13.481108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481099 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" containerName="payload-pre-processing" Sep 06 19:54:13.481275 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481153 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba5dff1-62a4-4fb0-9089-c55d30ec5f44" containerName="payload-pre-processing" Sep 06 19:54:13.481275 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481164 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="458ec939-67e3-4d12-8800-21245aab56ad" containerName="payload-processing" Sep 06 19:54:13.481275 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.481169 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4493b46e-cf02-44fe-95df-2f1cb3ba46d1" containerName="revoke-keys" Sep 06 19:54:13.483489 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.483474 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:13.486214 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.486157 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:54:13.486358 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.486235 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:54:13.490650 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.490626 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn"] Sep 06 19:54:13.521345 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.521320 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ae13a9-0ef1-42bb-99ba-94697f958b8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn\" (UID: \"69ae13a9-0ef1-42bb-99ba-94697f958b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:13.622061 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.621989 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ae13a9-0ef1-42bb-99ba-94697f958b8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn\" (UID: \"69ae13a9-0ef1-42bb-99ba-94697f958b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:13.622614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.622592 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ae13a9-0ef1-42bb-99ba-94697f958b8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn\" (UID: \"69ae13a9-0ef1-42bb-99ba-94697f958b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:13.636131 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.636106 2575 generic.go:356] "Generic (PLEG): container finished" podID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerID="e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f" exitCode=0 Sep 06 19:54:13.636257 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.636179 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" event={"ID":"7b404a2a-e6f3-4833-87ec-97c7031f428e","Type":"ContainerDied","Data":"e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f"} Sep 06 19:54:13.636587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.636573 2575 scope.go:117] "RemoveContainer" containerID="e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f" Sep 06 19:54:13.637679 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.637642 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:54:13.637811 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.637741 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:54:13.637885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.637820 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 06 19:54:13.796264 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:13.796216 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:14.011401 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.011323 2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 06 19:54:14.011401 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.011385 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 06 19:54:14.076715 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.076685 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn"] Sep 06 19:54:14.078224 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:54:14.078194 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ae13a9_0ef1_42bb_99ba_94697f958b8d.slice/crio-f54e3f3e2058256d888445e4f90ee509d814c9c2dc9c138c7852481eeab6ac53 WatchSource:0}: Error finding container f54e3f3e2058256d888445e4f90ee509d814c9c2dc9c138c7852481eeab6ac53: Status 404 returned error can't find the container with id f54e3f3e2058256d888445e4f90ee509d814c9c2dc9c138c7852481eeab6ac53 Sep 06 19:54:14.645332 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.645232 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" event={"ID":"7b404a2a-e6f3-4833-87ec-97c7031f428e","Type":"ContainerStarted","Data":"29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8"} Sep 06 19:54:14.646902 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.646875 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" event={"ID":"69ae13a9-0ef1-42bb-99ba-94697f958b8d","Type":"ContainerStarted","Data":"8d66f4df272a1504c1e764deffc5d55883b60408c3ab419b6d81611bd0a98fe6"} Sep 06 19:54:14.647016 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.646928 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" event={"ID":"69ae13a9-0ef1-42bb-99ba-94697f958b8d","Type":"ContainerStarted","Data":"f54e3f3e2058256d888445e4f90ee509d814c9c2dc9c138c7852481eeab6ac53"} Sep 06 19:54:14.677041 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:14.676993 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" podStartSLOduration=1.6769780220000001 podStartE2EDuration="1.676978022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:54:14.674591254 +0000 UTC m=+1597.409195986" watchObservedRunningTime="2026-09-06 19:54:14.676978022 +0000 UTC m=+1597.411582754" Sep 06 19:54:15.388967 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:15.388870 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:54:15.389193 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:15.389169 2575 patch_prober.go:28] interesting pod/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body= Sep 06 19:54:15.389239 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:15.389222 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" Sep 06 19:54:15.654019 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:15.653927 2575 generic.go:356] "Generic (PLEG): container finished" podID="69ae13a9-0ef1-42bb-99ba-94697f958b8d" containerID="8d66f4df272a1504c1e764deffc5d55883b60408c3ab419b6d81611bd0a98fe6" exitCode=0 Sep 06 19:54:15.654019 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:15.653986 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" event={"ID":"69ae13a9-0ef1-42bb-99ba-94697f958b8d","Type":"ContainerDied","Data":"8d66f4df272a1504c1e764deffc5d55883b60408c3ab419b6d81611bd0a98fe6"} Sep 06 19:54:16.389731 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:16.389696 2575 patch_prober.go:28] interesting pod/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body= Sep 06 19:54:16.389920 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:16.389770 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" Sep 06 19:54:16.839862 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:16.839831 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:16.955246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:16.955216 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ae13a9-0ef1-42bb-99ba-94697f958b8d-maas-api-service-ca\") pod \"69ae13a9-0ef1-42bb-99ba-94697f958b8d\" (UID: \"69ae13a9-0ef1-42bb-99ba-94697f958b8d\") " Sep 06 19:54:16.955561 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:16.955535 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ae13a9-0ef1-42bb-99ba-94697f958b8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69ae13a9-0ef1-42bb-99ba-94697f958b8d" (UID: "69ae13a9-0ef1-42bb-99ba-94697f958b8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:54:16.955637 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:16.955611 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ae13a9-0ef1-42bb-99ba-94697f958b8d-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:17.389522 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:17.389424 2575 patch_prober.go:28] interesting pod/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body= Sep 06 19:54:17.389522 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:17.389484 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" Sep 06 19:54:17.666854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:17.666829 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" Sep 06 19:54:17.666854 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:17.666841 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn" event={"ID":"69ae13a9-0ef1-42bb-99ba-94697f958b8d","Type":"ContainerDied","Data":"f54e3f3e2058256d888445e4f90ee509d814c9c2dc9c138c7852481eeab6ac53"} Sep 06 19:54:17.667047 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:17.666910 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f54e3f3e2058256d888445e4f90ee509d814c9c2dc9c138c7852481eeab6ac53" Sep 06 19:54:18.389693 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.389662 2575 patch_prober.go:28] interesting pod/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body= Sep 06 19:54:18.390201 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.389713 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" Sep 06 19:54:18.426988 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.426964 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:54:18.469042 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469011 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-data\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469211 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469119 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-podinfo\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469211 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469183 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-socket\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469223 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istiod-ca-cert\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469258 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:54:18.469337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469272 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-envoy\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469325 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-token\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469382 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg856\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-kube-api-access-kg856\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469434 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:54:18.469563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469441 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-certs\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469563 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469512 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-credential-socket\") pod \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\" (UID: \"c2480c06-9e85-4eca-98d2-d675c5bcb8dd\") " Sep 06 19:54:18.469853 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469703 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:54:18.470010 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.469880 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:54:18.470069 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.470017 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:54:18.470129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.470064 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.470129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.470082 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istiod-ca-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.470129 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.470097 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-workload-certs\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.470300 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.470114 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-data\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.471551 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.471515 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 19:54:18.471851 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.471821 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-kube-api-access-kg856" (OuterVolumeSpecName: "kube-api-access-kg856") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "kube-api-access-kg856". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:54:18.471997 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.471876 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:54:18.472115 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.472095 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2480c06-9e85-4eca-98d2-d675c5bcb8dd" (UID: "c2480c06-9e85-4eca-98d2-d675c5bcb8dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:54:18.572003 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.571969 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-envoy\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.572003 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.572004 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-token\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.572203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.572018 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg856\" (UniqueName: \"kubernetes.io/projected/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-kube-api-access-kg856\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.572203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.572032 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-credential-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.572203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.572046 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2480c06-9e85-4eca-98d2-d675c5bcb8dd-istio-podinfo\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:18.674184 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.674151 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerID="993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399" exitCode=0 Sep 06 19:54:18.674374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.674199 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" Sep 06 19:54:18.674374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.674195 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" event={"ID":"c2480c06-9e85-4eca-98d2-d675c5bcb8dd","Type":"ContainerDied","Data":"993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399"} Sep 06 19:54:18.674374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.674330 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd" event={"ID":"c2480c06-9e85-4eca-98d2-d675c5bcb8dd","Type":"ContainerDied","Data":"085717fba249a9303fd97ea3322e37876d4274490791d02ecfcc7f3945312f61"} Sep 06 19:54:18.674374 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.674360 2575 scope.go:117] "RemoveContainer" containerID="993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399" Sep 06 19:54:18.684962 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.684942 2575 scope.go:117] "RemoveContainer" containerID="c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577" Sep 06 19:54:18.715814 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.715697 2575 scope.go:117] "RemoveContainer" containerID="993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399" Sep 06 19:54:18.716250 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:54:18.716215 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399\": container with ID starting with 993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399 not found: ID does not exist" containerID="993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399" Sep 06 19:54:18.716377 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.716258 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399"} err="failed to get container status \"993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399\": rpc error: code = NotFound desc = could not find container \"993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399\": container with ID starting with 993199e746694cbea74306eb14947b1da58fa0b446af099f7647b7e9fa2ce399 not found: ID does not exist" Sep 06 19:54:18.716377 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.716284 2575 scope.go:117] "RemoveContainer" containerID="c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577" Sep 06 19:54:18.716576 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.716558 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd"] Sep 06 19:54:18.716691 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:54:18.716634 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577\": container with ID starting with c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577 not found: ID does not exist" containerID="c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577" Sep 06 19:54:18.716787 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.716741 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577"} err="failed to get container status \"c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577\": rpc error: code = NotFound desc = could not find container \"c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577\": container with ID starting with c92a685ec029a7feb88d7a17b63b92285d9d8660ed9c7d7e2d3119be3867c577 not found: ID does not exist" Sep 06 19:54:18.721240 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:18.721217 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b12ed2-openshift-default-b58b89f74-sp5jd"] Sep 06 19:54:19.389224 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:19.389182 2575 patch_prober.go:28] interesting pod/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body= Sep 06 19:54:19.389416 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:19.389273 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" Sep 06 19:54:19.763159 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:19.763127 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" path="/var/lib/kubelet/pods/c2480c06-9e85-4eca-98d2-d675c5bcb8dd/volumes" Sep 06 19:54:20.392594 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:20.392564 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:54:20.393018 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:20.392995 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:54:20.393783 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:20.393744 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:54:24.283309 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.283257 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll"] Sep 06 19:54:24.283794 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.283550 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" podUID="25c9977d-ba77-4250-af58-bc7790a66e44" containerName="payload-pre-processing" containerID="cri-o://b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d" gracePeriod=30 Sep 06 19:54:24.484103 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.484066 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn"] Sep 06 19:54:24.486843 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.486813 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ddeb5-clccn"] Sep 06 19:54:24.563203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.563173 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:54:24.636337 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.636303 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz8fg\" (UniqueName: \"kubernetes.io/projected/25c9977d-ba77-4250-af58-bc7790a66e44-kube-api-access-xz8fg\") pod \"25c9977d-ba77-4250-af58-bc7790a66e44\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " Sep 06 19:54:24.636522 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.636423 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25c9977d-ba77-4250-af58-bc7790a66e44-plugins-config-volume\") pod \"25c9977d-ba77-4250-af58-bc7790a66e44\" (UID: \"25c9977d-ba77-4250-af58-bc7790a66e44\") " Sep 06 19:54:24.636843 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.636819 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25c9977d-ba77-4250-af58-bc7790a66e44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "25c9977d-ba77-4250-af58-bc7790a66e44" (UID: "25c9977d-ba77-4250-af58-bc7790a66e44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:54:24.638523 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.638500 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c9977d-ba77-4250-af58-bc7790a66e44-kube-api-access-xz8fg" (OuterVolumeSpecName: "kube-api-access-xz8fg") pod "25c9977d-ba77-4250-af58-bc7790a66e44" (UID: "25c9977d-ba77-4250-af58-bc7790a66e44"). InnerVolumeSpecName "kube-api-access-xz8fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:54:24.715527 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.715485 2575 generic.go:356] "Generic (PLEG): container finished" podID="25c9977d-ba77-4250-af58-bc7790a66e44" containerID="b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d" exitCode=0 Sep 06 19:54:24.715527 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.715529 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" event={"ID":"25c9977d-ba77-4250-af58-bc7790a66e44","Type":"ContainerDied","Data":"b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d"} Sep 06 19:54:24.715800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.715532 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" Sep 06 19:54:24.715800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.715577 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll" event={"ID":"25c9977d-ba77-4250-af58-bc7790a66e44","Type":"ContainerDied","Data":"02d9aa3241e25dd84406a6eb8f7bad8350a145c7dfe3d984f2a3558e087fc01e"} Sep 06 19:54:24.715800 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.715603 2575 scope.go:117] "RemoveContainer" containerID="b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d" Sep 06 19:54:24.725643 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.725623 2575 scope.go:117] "RemoveContainer" containerID="b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d" Sep 06 19:54:24.726023 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:54:24.725999 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d\": container with ID starting with b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d not found: ID does not exist" containerID="b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d" Sep 06 19:54:24.726083 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.726031 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d"} err="failed to get container status \"b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d\": rpc error: code = NotFound desc = could not find container \"b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d\": container with ID starting with b294e522072a3d902a0a7c90832d12be5dbfe52311024b6daaa91035ec11a91d not found: ID does not exist" Sep 06 19:54:24.736865 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.736842 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll"] Sep 06 19:54:24.737362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.737309 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25c9977d-ba77-4250-af58-bc7790a66e44-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:24.737362 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.737341 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz8fg\" (UniqueName: \"kubernetes.io/projected/25c9977d-ba77-4250-af58-bc7790a66e44-kube-api-access-xz8fg\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:54:24.740294 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:24.740275 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ddeb5-76576b4554-n24ll"] Sep 06 19:54:25.764094 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:25.764061 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25c9977d-ba77-4250-af58-bc7790a66e44" path="/var/lib/kubelet/pods/25c9977d-ba77-4250-af58-bc7790a66e44/volumes" Sep 06 19:54:25.764470 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:25.764359 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ae13a9-0ef1-42bb-99ba-94697f958b8d" path="/var/lib/kubelet/pods/69ae13a9-0ef1-42bb-99ba-94697f958b8d/volumes" Sep 06 19:54:27.330737 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.330697 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2"] Sep 06 19:54:27.331334 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331313 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ae13a9-0ef1-42bb-99ba-94697f958b8d" containerName="revoke-keys" Sep 06 19:54:27.331423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331336 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ae13a9-0ef1-42bb-99ba-94697f958b8d" containerName="revoke-keys" Sep 06 19:54:27.331423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331351 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25c9977d-ba77-4250-af58-bc7790a66e44" containerName="payload-pre-processing" Sep 06 19:54:27.331423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331361 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="25c9977d-ba77-4250-af58-bc7790a66e44" containerName="payload-pre-processing" Sep 06 19:54:27.331423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331378 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" Sep 06 19:54:27.331423 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331388 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" Sep 06 19:54:27.331675 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331484 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" Sep 06 19:54:27.331675 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331501 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="25c9977d-ba77-4250-af58-bc7790a66e44" containerName="payload-pre-processing" Sep 06 19:54:27.331675 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.331514 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ae13a9-0ef1-42bb-99ba-94697f958b8d" containerName="revoke-keys" Sep 06 19:54:27.336096 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.336075 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.338468 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.338445 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-adaf11\"" Sep 06 19:54:27.338566 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.338473 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-adaf11-dockercfg-7zz2c\"" Sep 06 19:54:27.344515 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.344493 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2"] Sep 06 19:54:27.388502 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.388466 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f"] Sep 06 19:54:27.388892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.388877 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" Sep 06 19:54:27.388892 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.388893 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" Sep 06 19:54:27.388995 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.388962 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2480c06-9e85-4eca-98d2-d675c5bcb8dd" containerName="istio-proxy" Sep 06 19:54:27.392078 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.392063 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.402600 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.402569 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f"] Sep 06 19:54:27.465628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.465593 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrd5p\" (UniqueName: \"kubernetes.io/projected/b1d395a0-c72b-4264-bfaf-c7ab376768e8-kube-api-access-rrd5p\") pod \"payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.465885 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.465860 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnc5r\" (UniqueName: \"kubernetes.io/projected/cb1507c2-d633-4519-845e-72986af14d2d-kube-api-access-dnc5r\") pod \"payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.465985 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.465962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d395a0-c72b-4264-bfaf-c7ab376768e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.466090 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.466073 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1507c2-d633-4519-845e-72986af14d2d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.566911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.566869 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1507c2-d633-4519-845e-72986af14d2d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.566911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.566912 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrd5p\" (UniqueName: \"kubernetes.io/projected/b1d395a0-c72b-4264-bfaf-c7ab376768e8-kube-api-access-rrd5p\") pod \"payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.567139 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.566970 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnc5r\" (UniqueName: \"kubernetes.io/projected/cb1507c2-d633-4519-845e-72986af14d2d-kube-api-access-dnc5r\") pod \"payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.567139 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.567012 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d395a0-c72b-4264-bfaf-c7ab376768e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.567580 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.567558 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1507c2-d633-4519-845e-72986af14d2d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.567709 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.567687 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d395a0-c72b-4264-bfaf-c7ab376768e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.574984 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.574958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrd5p\" (UniqueName: \"kubernetes.io/projected/b1d395a0-c72b-4264-bfaf-c7ab376768e8-kube-api-access-rrd5p\") pod \"payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.575104 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.574958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnc5r\" (UniqueName: \"kubernetes.io/projected/cb1507c2-d633-4519-845e-72986af14d2d-kube-api-access-dnc5r\") pod \"payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.648135 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.648048 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:27.704113 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.704081 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:27.920442 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.920417 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2"] Sep 06 19:54:27.921468 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:54:27.921433 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1507c2_d633_4519_845e_72986af14d2d.slice/crio-56cc3b8d3469dfef937973ee82530fb25e51779122838f295190a2801a16d99e WatchSource:0}: Error finding container 56cc3b8d3469dfef937973ee82530fb25e51779122838f295190a2801a16d99e: Status 404 returned error can't find the container with id 56cc3b8d3469dfef937973ee82530fb25e51779122838f295190a2801a16d99e Sep 06 19:54:27.989678 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:27.989652 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f"] Sep 06 19:54:27.991567 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:54:27.991542 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d395a0_c72b_4264_bfaf_c7ab376768e8.slice/crio-f6083c808f1891319bb5c08c42b63a350fc16c022a3f6bb077c15ea10d20d3ba WatchSource:0}: Error finding container f6083c808f1891319bb5c08c42b63a350fc16c022a3f6bb077c15ea10d20d3ba: Status 404 returned error can't find the container with id f6083c808f1891319bb5c08c42b63a350fc16c022a3f6bb077c15ea10d20d3ba Sep 06 19:54:28.751316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.751229 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" event={"ID":"b1d395a0-c72b-4264-bfaf-c7ab376768e8","Type":"ContainerStarted","Data":"677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877"} Sep 06 19:54:28.751316 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.751298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" event={"ID":"b1d395a0-c72b-4264-bfaf-c7ab376768e8","Type":"ContainerStarted","Data":"f6083c808f1891319bb5c08c42b63a350fc16c022a3f6bb077c15ea10d20d3ba"} Sep 06 19:54:28.751796 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.751328 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:28.752820 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.752792 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" event={"ID":"cb1507c2-d633-4519-845e-72986af14d2d","Type":"ContainerStarted","Data":"1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e"} Sep 06 19:54:28.752949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.752837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" event={"ID":"cb1507c2-d633-4519-845e-72986af14d2d","Type":"ContainerStarted","Data":"56cc3b8d3469dfef937973ee82530fb25e51779122838f295190a2801a16d99e"} Sep 06 19:54:28.752949 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.752859 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:28.773619 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.773576 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" podStartSLOduration=1.773561894 podStartE2EDuration="1.773561894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:54:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:54:28.772189595 +0000 UTC m=+1611.506794325" watchObservedRunningTime="2026-09-06 19:54:28.773561894 +0000 UTC m=+1611.508166657" Sep 06 19:54:28.800158 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:28.800115 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" podStartSLOduration=1.800100151 podStartE2EDuration="1.800100151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:54:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:54:28.798939512 +0000 UTC m=+1611.533544244" watchObservedRunningTime="2026-09-06 19:54:28.800100151 +0000 UTC m=+1611.534704909" Sep 06 19:54:39.762957 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:39.762925 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:54:39.763326 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:39.762986 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:54:55.325787 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:55.325735 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dc6h2_881dd41f-b5f1-4930-8e55-a219f83580f1/payload-processing/0.log" Sep 06 19:54:55.437443 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:54:55.437406 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f_b1d395a0-c72b-4264-bfaf-c7ab376768e8/payload-processing/0.log" Sep 06 19:55:51.387877 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:55:51.387850 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt_938dfedc-198e-4c01-b083-1ad7918e638b/payload-processing/0.log" Sep 06 19:55:51.498137 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:55:51.498105 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dc6h2_881dd41f-b5f1-4930-8e55-a219f83580f1/payload-processing/0.log" Sep 06 19:55:51.609287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:55:51.609251 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f_b1d395a0-c72b-4264-bfaf-c7ab376768e8/payload-processing/0.log" Sep 06 19:56:20.567170 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.567133 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62"] Sep 06 19:56:20.570348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.570328 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:20.572841 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.572821 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:56:20.572967 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.572864 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:56:20.577743 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.577723 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62"] Sep 06 19:56:20.661342 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.661298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb61b4a-0919-4349-9cee-407a2683c60b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62\" (UID: \"2bb61b4a-0919-4349-9cee-407a2683c60b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:20.762448 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.762418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb61b4a-0919-4349-9cee-407a2683c60b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62\" (UID: \"2bb61b4a-0919-4349-9cee-407a2683c60b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:20.763090 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.763062 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb61b4a-0919-4349-9cee-407a2683c60b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62\" (UID: \"2bb61b4a-0919-4349-9cee-407a2683c60b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:20.882418 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:20.882330 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:21.152917 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:21.152828 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62"] Sep 06 19:56:21.156336 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:56:21.156304 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb61b4a_0919_4349_9cee_407a2683c60b.slice/crio-abf195f1da181421cf46da6b08eb411f50f77f293269c655b8c85ddee31f0d05 WatchSource:0}: Error finding container abf195f1da181421cf46da6b08eb411f50f77f293269c655b8c85ddee31f0d05: Status 404 returned error can't find the container with id abf195f1da181421cf46da6b08eb411f50f77f293269c655b8c85ddee31f0d05 Sep 06 19:56:21.421108 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:21.421078 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" event={"ID":"2bb61b4a-0919-4349-9cee-407a2683c60b","Type":"ContainerStarted","Data":"abf195f1da181421cf46da6b08eb411f50f77f293269c655b8c85ddee31f0d05"} Sep 06 19:56:22.428899 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:22.428860 2575 generic.go:356] "Generic (PLEG): container finished" podID="2bb61b4a-0919-4349-9cee-407a2683c60b" containerID="056476d18576dddcc130bc516b6bdf763903bae740e5b85611c1db39b2ed78ca" exitCode=0 Sep 06 19:56:22.429364 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:22.428946 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" event={"ID":"2bb61b4a-0919-4349-9cee-407a2683c60b","Type":"ContainerDied","Data":"056476d18576dddcc130bc516b6bdf763903bae740e5b85611c1db39b2ed78ca"} Sep 06 19:56:23.573677 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:23.573649 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:23.692057 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:23.691980 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb61b4a-0919-4349-9cee-407a2683c60b-maas-api-service-ca\") pod \"2bb61b4a-0919-4349-9cee-407a2683c60b\" (UID: \"2bb61b4a-0919-4349-9cee-407a2683c60b\") " Sep 06 19:56:23.692383 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:23.692363 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb61b4a-0919-4349-9cee-407a2683c60b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2bb61b4a-0919-4349-9cee-407a2683c60b" (UID: "2bb61b4a-0919-4349-9cee-407a2683c60b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:56:23.793386 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:23.793345 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb61b4a-0919-4349-9cee-407a2683c60b-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:56:24.442231 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:24.442189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" event={"ID":"2bb61b4a-0919-4349-9cee-407a2683c60b","Type":"ContainerDied","Data":"abf195f1da181421cf46da6b08eb411f50f77f293269c655b8c85ddee31f0d05"} Sep 06 19:56:24.442231 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:24.442224 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62" Sep 06 19:56:24.442445 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:24.442247 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abf195f1da181421cf46da6b08eb411f50f77f293269c655b8c85ddee31f0d05" Sep 06 19:56:26.726024 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:26.725979 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62"] Sep 06 19:56:26.728630 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:26.728600 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c95664-25n62"] Sep 06 19:56:27.763765 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:27.763719 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bb61b4a-0919-4349-9cee-407a2683c60b" path="/var/lib/kubelet/pods/2bb61b4a-0919-4349-9cee-407a2683c60b/volumes" Sep 06 19:56:36.601054 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.601020 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg"] Sep 06 19:56:36.601607 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.601583 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb61b4a-0919-4349-9cee-407a2683c60b" containerName="revoke-keys" Sep 06 19:56:36.601607 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.601607 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb61b4a-0919-4349-9cee-407a2683c60b" containerName="revoke-keys" Sep 06 19:56:36.601868 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.601686 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb61b4a-0919-4349-9cee-407a2683c60b" containerName="revoke-keys" Sep 06 19:56:36.605184 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.605167 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:36.607736 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.607710 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:56:36.608004 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.607985 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:56:36.610007 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.609982 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg"] Sep 06 19:56:36.620765 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.620724 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028f45f-cb2e-488b-8894-a6e77d6ad442-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg\" (UID: \"0028f45f-cb2e-488b-8894-a6e77d6ad442\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:36.721511 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.721472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028f45f-cb2e-488b-8894-a6e77d6ad442-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg\" (UID: \"0028f45f-cb2e-488b-8894-a6e77d6ad442\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:36.722158 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.722139 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028f45f-cb2e-488b-8894-a6e77d6ad442-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg\" (UID: \"0028f45f-cb2e-488b-8894-a6e77d6ad442\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:36.917329 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:36.917291 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:37.188071 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:37.188045 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg"] Sep 06 19:56:37.189683 ip-10-0-135-236 kubenswrapper[2575]: W0906 19:56:37.189658 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0028f45f_cb2e_488b_8894_a6e77d6ad442.slice/crio-af8d47d5e9d02010cc191f192736529181de16f10383866b60febc351878bd17 WatchSource:0}: Error finding container af8d47d5e9d02010cc191f192736529181de16f10383866b60febc351878bd17: Status 404 returned error can't find the container with id af8d47d5e9d02010cc191f192736529181de16f10383866b60febc351878bd17 Sep 06 19:56:37.518490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:37.518454 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" event={"ID":"0028f45f-cb2e-488b-8894-a6e77d6ad442","Type":"ContainerStarted","Data":"bfa5fe9e480028d1a91a179a497c4ebc8c3847938b66f32618717a8e121f23ab"} Sep 06 19:56:37.518628 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:37.518510 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" event={"ID":"0028f45f-cb2e-488b-8894-a6e77d6ad442","Type":"ContainerStarted","Data":"af8d47d5e9d02010cc191f192736529181de16f10383866b60febc351878bd17"} Sep 06 19:56:37.531370 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:37.531315 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" podStartSLOduration=1.5312942280000001 podStartE2EDuration="1.531294228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:56:37.531130622 +0000 UTC m=+1740.265735354" watchObservedRunningTime="2026-09-06 19:56:37.531294228 +0000 UTC m=+1740.265898961" Sep 06 19:56:38.051578 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:38.048673 2575 scope.go:117] "RemoveContainer" containerID="27c4efbe59f36e07cb5a1afc3a75df73b04d4d1465e9c40e3be6086ca2f90e11" Sep 06 19:56:38.525908 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:38.525869 2575 generic.go:356] "Generic (PLEG): container finished" podID="0028f45f-cb2e-488b-8894-a6e77d6ad442" containerID="bfa5fe9e480028d1a91a179a497c4ebc8c3847938b66f32618717a8e121f23ab" exitCode=0 Sep 06 19:56:38.526091 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:38.525960 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" event={"ID":"0028f45f-cb2e-488b-8894-a6e77d6ad442","Type":"ContainerDied","Data":"bfa5fe9e480028d1a91a179a497c4ebc8c3847938b66f32618717a8e121f23ab"} Sep 06 19:56:39.670727 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:39.670703 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:39.744584 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:39.744544 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028f45f-cb2e-488b-8894-a6e77d6ad442-maas-api-service-ca\") pod \"0028f45f-cb2e-488b-8894-a6e77d6ad442\" (UID: \"0028f45f-cb2e-488b-8894-a6e77d6ad442\") " Sep 06 19:56:39.744940 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:39.744915 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0028f45f-cb2e-488b-8894-a6e77d6ad442-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0028f45f-cb2e-488b-8894-a6e77d6ad442" (UID: "0028f45f-cb2e-488b-8894-a6e77d6ad442"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:56:39.845955 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:39.845866 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028f45f-cb2e-488b-8894-a6e77d6ad442-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:56:40.539777 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:40.539722 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" Sep 06 19:56:40.539973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:40.539778 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg" event={"ID":"0028f45f-cb2e-488b-8894-a6e77d6ad442","Type":"ContainerDied","Data":"af8d47d5e9d02010cc191f192736529181de16f10383866b60febc351878bd17"} Sep 06 19:56:40.539973 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:40.539836 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af8d47d5e9d02010cc191f192736529181de16f10383866b60febc351878bd17" Sep 06 19:56:52.118372 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.118330 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt"] Sep 06 19:56:52.118924 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.118618 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" podUID="938dfedc-198e-4c01-b083-1ad7918e638b" containerName="payload-processing" containerID="cri-o://c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac" gracePeriod=30 Sep 06 19:56:52.260490 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.260157 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg"] Sep 06 19:56:52.263200 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.263174 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a30d6c-6sxvg"] Sep 06 19:56:52.398621 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.398594 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:56:52.470385 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.470353 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgljl\" (UniqueName: \"kubernetes.io/projected/938dfedc-198e-4c01-b083-1ad7918e638b-kube-api-access-sgljl\") pod \"938dfedc-198e-4c01-b083-1ad7918e638b\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " Sep 06 19:56:52.470570 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.470429 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938dfedc-198e-4c01-b083-1ad7918e638b-plugins-config-volume\") pod \"938dfedc-198e-4c01-b083-1ad7918e638b\" (UID: \"938dfedc-198e-4c01-b083-1ad7918e638b\") " Sep 06 19:56:52.470785 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.470734 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/938dfedc-198e-4c01-b083-1ad7918e638b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "938dfedc-198e-4c01-b083-1ad7918e638b" (UID: "938dfedc-198e-4c01-b083-1ad7918e638b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:56:52.472614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.472592 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/938dfedc-198e-4c01-b083-1ad7918e638b-kube-api-access-sgljl" (OuterVolumeSpecName: "kube-api-access-sgljl") pod "938dfedc-198e-4c01-b083-1ad7918e638b" (UID: "938dfedc-198e-4c01-b083-1ad7918e638b"). InnerVolumeSpecName "kube-api-access-sgljl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:56:52.571581 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.571542 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938dfedc-198e-4c01-b083-1ad7918e638b-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:56:52.571581 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.571576 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgljl\" (UniqueName: \"kubernetes.io/projected/938dfedc-198e-4c01-b083-1ad7918e638b-kube-api-access-sgljl\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:56:52.613269 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.613231 2575 generic.go:356] "Generic (PLEG): container finished" podID="938dfedc-198e-4c01-b083-1ad7918e638b" containerID="c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac" exitCode=0 Sep 06 19:56:52.613477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.613276 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" Sep 06 19:56:52.613477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.613318 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" event={"ID":"938dfedc-198e-4c01-b083-1ad7918e638b","Type":"ContainerDied","Data":"c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac"} Sep 06 19:56:52.613477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.613370 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt" event={"ID":"938dfedc-198e-4c01-b083-1ad7918e638b","Type":"ContainerDied","Data":"6d7128d88dbd7c3b38ae032a2a59fde514174309c9f5957eb64b8a1ca3cd0a86"} Sep 06 19:56:52.613477 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.613393 2575 scope.go:117] "RemoveContainer" containerID="c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac" Sep 06 19:56:52.623898 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.623880 2575 scope.go:117] "RemoveContainer" containerID="c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac" Sep 06 19:56:52.624198 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:56:52.624178 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac\": container with ID starting with c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac not found: ID does not exist" containerID="c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac" Sep 06 19:56:52.624272 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.624207 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac"} err="failed to get container status \"c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac\": rpc error: code = NotFound desc = could not find container \"c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac\": container with ID starting with c0ebf3d878e2a653737c81b8b2c2a0cf33b87286c4f241fde1a6db1b3f9a23ac not found: ID does not exist" Sep 06 19:56:52.635514 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.635485 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt"] Sep 06 19:56:52.640761 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:52.640728 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a30d6c-6895fd5cf-bjnlt"] Sep 06 19:56:53.762816 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:53.762785 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0028f45f-cb2e-488b-8894-a6e77d6ad442" path="/var/lib/kubelet/pods/0028f45f-cb2e-488b-8894-a6e77d6ad442/volumes" Sep 06 19:56:53.763425 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:56:53.763289 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="938dfedc-198e-4c01-b083-1ad7918e638b" path="/var/lib/kubelet/pods/938dfedc-198e-4c01-b083-1ad7918e638b/volumes" Sep 06 19:57:37.841976 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:37.841946 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:57:37.845962 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:37.845942 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 19:57:40.374648 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.374611 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p"] Sep 06 19:57:40.375094 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.374945 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" containerID="cri-o://29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8" gracePeriod=30 Sep 06 19:57:40.744864 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.744829 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5"] Sep 06 19:57:40.745227 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.745213 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0028f45f-cb2e-488b-8894-a6e77d6ad442" containerName="revoke-keys" Sep 06 19:57:40.745277 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.745231 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0028f45f-cb2e-488b-8894-a6e77d6ad442" containerName="revoke-keys" Sep 06 19:57:40.745277 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.745266 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="938dfedc-198e-4c01-b083-1ad7918e638b" containerName="payload-processing" Sep 06 19:57:40.745277 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.745274 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="938dfedc-198e-4c01-b083-1ad7918e638b" containerName="payload-processing" Sep 06 19:57:40.745392 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.745377 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0028f45f-cb2e-488b-8894-a6e77d6ad442" containerName="revoke-keys" Sep 06 19:57:40.745427 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.745392 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="938dfedc-198e-4c01-b083-1ad7918e638b" containerName="payload-processing" Sep 06 19:57:40.748508 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.748492 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:40.751015 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.750986 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 19:57:40.751198 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.751048 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 19:57:40.754888 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.754861 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5"] Sep 06 19:57:40.836526 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.836467 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1296006-07b9-4608-b8a1-839eb0be31dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5\" (UID: \"a1296006-07b9-4608-b8a1-839eb0be31dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:40.937541 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.937509 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1296006-07b9-4608-b8a1-839eb0be31dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5\" (UID: \"a1296006-07b9-4608-b8a1-839eb0be31dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:40.938133 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:40.938113 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1296006-07b9-4608-b8a1-839eb0be31dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5\" (UID: \"a1296006-07b9-4608-b8a1-839eb0be31dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:41.060933 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:41.060849 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:41.336187 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:41.336157 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5"] Sep 06 19:57:41.914141 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:41.914092 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" event={"ID":"a1296006-07b9-4608-b8a1-839eb0be31dc","Type":"ContainerStarted","Data":"c28312f68d4e8f1a7e7c05102497dccb0df1335b7c1c4fea84e20f750c974566"} Sep 06 19:57:41.914533 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:41.914189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" event={"ID":"a1296006-07b9-4608-b8a1-839eb0be31dc","Type":"ContainerStarted","Data":"839b64741bdfc57b426f6b4339b4ffb3ec6993b0e7265c11156e997749a93b60"} Sep 06 19:57:41.929549 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:41.929492 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" podStartSLOduration=1.929477902 podStartE2EDuration="1.929477902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 19:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 19:57:41.92721315 +0000 UTC m=+1804.661817883" watchObservedRunningTime="2026-09-06 19:57:41.929477902 +0000 UTC m=+1804.664082634" Sep 06 19:57:42.922175 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:42.922138 2575 generic.go:356] "Generic (PLEG): container finished" podID="a1296006-07b9-4608-b8a1-839eb0be31dc" containerID="c28312f68d4e8f1a7e7c05102497dccb0df1335b7c1c4fea84e20f750c974566" exitCode=0 Sep 06 19:57:42.922533 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:42.922221 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" event={"ID":"a1296006-07b9-4608-b8a1-839eb0be31dc","Type":"ContainerDied","Data":"c28312f68d4e8f1a7e7c05102497dccb0df1335b7c1c4fea84e20f750c974566"} Sep 06 19:57:44.071399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.071373 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:44.171918 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.171887 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1296006-07b9-4608-b8a1-839eb0be31dc-maas-api-service-ca\") pod \"a1296006-07b9-4608-b8a1-839eb0be31dc\" (UID: \"a1296006-07b9-4608-b8a1-839eb0be31dc\") " Sep 06 19:57:44.172283 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.172244 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1296006-07b9-4608-b8a1-839eb0be31dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1296006-07b9-4608-b8a1-839eb0be31dc" (UID: "a1296006-07b9-4608-b8a1-839eb0be31dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:57:44.273636 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.273552 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1296006-07b9-4608-b8a1-839eb0be31dc-maas-api-service-ca\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:44.938409 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.938383 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" Sep 06 19:57:44.938582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.938418 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5" event={"ID":"a1296006-07b9-4608-b8a1-839eb0be31dc","Type":"ContainerDied","Data":"839b64741bdfc57b426f6b4339b4ffb3ec6993b0e7265c11156e997749a93b60"} Sep 06 19:57:44.938582 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:44.938477 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="839b64741bdfc57b426f6b4339b4ffb3ec6993b0e7265c11156e997749a93b60" Sep 06 19:57:45.642394 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.642336 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:57:45.786168 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786136 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-certs\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786168 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786175 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-envoy\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786206 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b404a2a-e6f3-4833-87ec-97c7031f428e-istiod-ca-cert\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786237 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-credential-socket\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786293 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-token\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786314 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-data\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786343 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pxhb\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-kube-api-access-2pxhb\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786412 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786395 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-podinfo\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786725 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786434 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-socket\") pod \"7b404a2a-e6f3-4833-87ec-97c7031f428e\" (UID: \"7b404a2a-e6f3-4833-87ec-97c7031f428e\") " Sep 06 19:57:45.786725 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786443 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:57:45.786725 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786599 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:57:45.786725 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786676 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:57:45.786725 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786704 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b404a2a-e6f3-4833-87ec-97c7031f428e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:57:45.787009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786832 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:57:45.787009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786888 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-certs\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.787009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786910 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b404a2a-e6f3-4833-87ec-97c7031f428e-istiod-ca-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.787009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786923 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-credential-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.787009 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.786937 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-data\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.788702 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.788671 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:57:45.788847 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.788701 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-kube-api-access-2pxhb" (OuterVolumeSpecName: "kube-api-access-2pxhb") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "kube-api-access-2pxhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:57:45.788847 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.788831 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 19:57:45.789104 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.789077 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b404a2a-e6f3-4833-87ec-97c7031f428e" (UID: "7b404a2a-e6f3-4833-87ec-97c7031f428e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:57:45.888437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.888397 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-envoy\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.888437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.888426 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-token\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.888437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.888437 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pxhb\" (UniqueName: \"kubernetes.io/projected/7b404a2a-e6f3-4833-87ec-97c7031f428e-kube-api-access-2pxhb\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.888437 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.888445 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b404a2a-e6f3-4833-87ec-97c7031f428e-istio-podinfo\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.888697 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.888455 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b404a2a-e6f3-4833-87ec-97c7031f428e-workload-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:45.946034 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.945998 2575 generic.go:356] "Generic (PLEG): container finished" podID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerID="29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8" exitCode=0 Sep 06 19:57:45.946218 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.946066 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" Sep 06 19:57:45.946218 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.946086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" event={"ID":"7b404a2a-e6f3-4833-87ec-97c7031f428e","Type":"ContainerDied","Data":"29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8"} Sep 06 19:57:45.946218 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.946142 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p" event={"ID":"7b404a2a-e6f3-4833-87ec-97c7031f428e","Type":"ContainerDied","Data":"02e712a952f99e6422fae4f2a0d7d61465f48a9dbcbec7a35b3e1b55235d4d1f"} Sep 06 19:57:45.946218 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.946162 2575 scope.go:117] "RemoveContainer" containerID="29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8" Sep 06 19:57:45.956153 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.956136 2575 scope.go:117] "RemoveContainer" containerID="e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f" Sep 06 19:57:45.964883 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.964864 2575 scope.go:117] "RemoveContainer" containerID="29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8" Sep 06 19:57:45.965324 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:57:45.965287 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8\": container with ID starting with 29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8 not found: ID does not exist" containerID="29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8" Sep 06 19:57:45.965508 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.965473 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8"} err="failed to get container status \"29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8\": rpc error: code = NotFound desc = could not find container \"29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8\": container with ID starting with 29c5df43cd10949fa65737e8a8ecfe65eb8bad13085105d727a6a402edc67da8 not found: ID does not exist" Sep 06 19:57:45.965508 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.965509 2575 scope.go:117] "RemoveContainer" containerID="e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f" Sep 06 19:57:45.965990 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:57:45.965917 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f\": container with ID starting with e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f not found: ID does not exist" containerID="e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f" Sep 06 19:57:45.965990 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.965960 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f"} err="failed to get container status \"e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f\": rpc error: code = NotFound desc = could not find container \"e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f\": container with ID starting with e0e7bb2c84296ea0b68b0950b3331a839e5689b245d4fab5c6f3bab2dda7a83f not found: ID does not exist" Sep 06 19:57:45.969287 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.969266 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p"] Sep 06 19:57:45.971632 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:45.971601 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-a30d6c-openshift-default-6667d8944d-78r4p"] Sep 06 19:57:47.763583 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:47.763549 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" path="/var/lib/kubelet/pods/7b404a2a-e6f3-4833-87ec-97c7031f428e/volumes" Sep 06 19:57:51.158258 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.158225 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f"] Sep 06 19:57:51.159043 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.159006 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" podUID="b1d395a0-c72b-4264-bfaf-c7ab376768e8" containerName="payload-processing" containerID="cri-o://677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877" gracePeriod=30 Sep 06 19:57:51.162674 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.162646 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2"] Sep 06 19:57:51.164111 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.164077 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" podUID="cb1507c2-d633-4519-845e-72986af14d2d" containerName="payload-pre-processing" containerID="cri-o://1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e" gracePeriod=30 Sep 06 19:57:51.299363 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.299324 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5"] Sep 06 19:57:51.303356 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.303317 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-adaf11-cqpf5"] Sep 06 19:57:51.436992 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.436967 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:57:51.456004 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.455976 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:57:51.541810 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.541773 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d395a0-c72b-4264-bfaf-c7ab376768e8-plugins-config-volume\") pod \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " Sep 06 19:57:51.541978 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.541868 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrd5p\" (UniqueName: \"kubernetes.io/projected/b1d395a0-c72b-4264-bfaf-c7ab376768e8-kube-api-access-rrd5p\") pod \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\" (UID: \"b1d395a0-c72b-4264-bfaf-c7ab376768e8\") " Sep 06 19:57:51.541978 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.541937 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnc5r\" (UniqueName: \"kubernetes.io/projected/cb1507c2-d633-4519-845e-72986af14d2d-kube-api-access-dnc5r\") pod \"cb1507c2-d633-4519-845e-72986af14d2d\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " Sep 06 19:57:51.541978 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.541962 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1507c2-d633-4519-845e-72986af14d2d-plugins-config-volume\") pod \"cb1507c2-d633-4519-845e-72986af14d2d\" (UID: \"cb1507c2-d633-4519-845e-72986af14d2d\") " Sep 06 19:57:51.542165 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.542143 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d395a0-c72b-4264-bfaf-c7ab376768e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1d395a0-c72b-4264-bfaf-c7ab376768e8" (UID: "b1d395a0-c72b-4264-bfaf-c7ab376768e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:57:51.542395 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.542363 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1507c2-d633-4519-845e-72986af14d2d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb1507c2-d633-4519-845e-72986af14d2d" (UID: "cb1507c2-d633-4519-845e-72986af14d2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:57:51.544222 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.544200 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb1507c2-d633-4519-845e-72986af14d2d-kube-api-access-dnc5r" (OuterVolumeSpecName: "kube-api-access-dnc5r") pod "cb1507c2-d633-4519-845e-72986af14d2d" (UID: "cb1507c2-d633-4519-845e-72986af14d2d"). InnerVolumeSpecName "kube-api-access-dnc5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:57:51.544222 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.544207 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d395a0-c72b-4264-bfaf-c7ab376768e8-kube-api-access-rrd5p" (OuterVolumeSpecName: "kube-api-access-rrd5p") pod "b1d395a0-c72b-4264-bfaf-c7ab376768e8" (UID: "b1d395a0-c72b-4264-bfaf-c7ab376768e8"). InnerVolumeSpecName "kube-api-access-rrd5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:57:51.642879 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.642847 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d395a0-c72b-4264-bfaf-c7ab376768e8-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:51.642879 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.642874 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrd5p\" (UniqueName: \"kubernetes.io/projected/b1d395a0-c72b-4264-bfaf-c7ab376768e8-kube-api-access-rrd5p\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:51.642879 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.642887 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnc5r\" (UniqueName: \"kubernetes.io/projected/cb1507c2-d633-4519-845e-72986af14d2d-kube-api-access-dnc5r\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:51.643109 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.642895 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1507c2-d633-4519-845e-72986af14d2d-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:57:51.762554 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.762475 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1296006-07b9-4608-b8a1-839eb0be31dc" path="/var/lib/kubelet/pods/a1296006-07b9-4608-b8a1-839eb0be31dc/volumes" Sep 06 19:57:51.982895 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.982859 2575 generic.go:356] "Generic (PLEG): container finished" podID="b1d395a0-c72b-4264-bfaf-c7ab376768e8" containerID="677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877" exitCode=0 Sep 06 19:57:51.983086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.982915 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" Sep 06 19:57:51.983086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.982946 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" event={"ID":"b1d395a0-c72b-4264-bfaf-c7ab376768e8","Type":"ContainerDied","Data":"677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877"} Sep 06 19:57:51.983086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.983004 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f" event={"ID":"b1d395a0-c72b-4264-bfaf-c7ab376768e8","Type":"ContainerDied","Data":"f6083c808f1891319bb5c08c42b63a350fc16c022a3f6bb077c15ea10d20d3ba"} Sep 06 19:57:51.983086 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.983028 2575 scope.go:117] "RemoveContainer" containerID="677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877" Sep 06 19:57:51.984542 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.984515 2575 generic.go:356] "Generic (PLEG): container finished" podID="cb1507c2-d633-4519-845e-72986af14d2d" containerID="1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e" exitCode=0 Sep 06 19:57:51.984657 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.984547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" event={"ID":"cb1507c2-d633-4519-845e-72986af14d2d","Type":"ContainerDied","Data":"1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e"} Sep 06 19:57:51.984657 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.984578 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" Sep 06 19:57:51.984657 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.984601 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2" event={"ID":"cb1507c2-d633-4519-845e-72986af14d2d","Type":"ContainerDied","Data":"56cc3b8d3469dfef937973ee82530fb25e51779122838f295190a2801a16d99e"} Sep 06 19:57:51.993004 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.992988 2575 scope.go:117] "RemoveContainer" containerID="677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877" Sep 06 19:57:51.993280 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:57:51.993261 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877\": container with ID starting with 677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877 not found: ID does not exist" containerID="677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877" Sep 06 19:57:51.993360 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.993288 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877"} err="failed to get container status \"677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877\": rpc error: code = NotFound desc = could not find container \"677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877\": container with ID starting with 677a8a382faed7a4747782854b8f00ab9af0c96cb05d94802a8e101ac6748877 not found: ID does not exist" Sep 06 19:57:51.993360 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:51.993314 2575 scope.go:117] "RemoveContainer" containerID="1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e" Sep 06 19:57:52.005403 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:52.005379 2575 scope.go:117] "RemoveContainer" containerID="1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e" Sep 06 19:57:52.005589 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:52.005568 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f"] Sep 06 19:57:52.005774 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:57:52.005730 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e\": container with ID starting with 1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e not found: ID does not exist" containerID="1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e" Sep 06 19:57:52.005847 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:52.005787 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e"} err="failed to get container status \"1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e\": rpc error: code = NotFound desc = could not find container \"1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e\": container with ID starting with 1c7b07606109a1a18a647b8cf1ef329ef68d53c13f3802d3505dcb6272b2538e not found: ID does not exist" Sep 06 19:57:52.007986 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:52.007966 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-adaf11-5cbf4c47dc-v4s7f"] Sep 06 19:57:52.018246 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:52.018179 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2"] Sep 06 19:57:52.021766 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:52.021718 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-adaf11-69fbcf78c4-xrmx2"] Sep 06 19:57:53.762179 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:53.762138 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d395a0-c72b-4264-bfaf-c7ab376768e8" path="/var/lib/kubelet/pods/b1d395a0-c72b-4264-bfaf-c7ab376768e8/volumes" Sep 06 19:57:53.762680 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:57:53.762619 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb1507c2-d633-4519-845e-72986af14d2d" path="/var/lib/kubelet/pods/cb1507c2-d633-4519-845e-72986af14d2d/volumes" Sep 06 19:58:15.388351 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:15.388313 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx"] Sep 06 19:58:15.388962 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:15.388584 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" containerID="cri-o://b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73" gracePeriod=30 Sep 06 19:58:20.650495 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.650461 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:58:20.713964 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.713934 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5ffv\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-kube-api-access-n5ffv\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.713964 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.713966 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-credential-socket\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714174 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714152 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11195ef4-b947-4745-861a-9e9324abd263-istio-podinfo\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714229 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714187 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-data\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714229 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714209 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-istio-token\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714247 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-socket\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714278 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11195ef4-b947-4745-861a-9e9324abd263-istiod-ca-cert\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714230 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:58:20.714348 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714326 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-certs\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714571 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714384 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-envoy\") pod \"11195ef4-b947-4745-861a-9e9324abd263\" (UID: \"11195ef4-b947-4745-861a-9e9324abd263\") " Sep 06 19:58:20.714571 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714485 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-data" (OuterVolumeSpecName: "istio-data") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:58:20.714571 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714521 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:58:20.714738 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714719 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:58:20.714828 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714816 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-credential-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.715052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714837 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-data\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.715052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714853 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-socket\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.715052 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.714932 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11195ef4-b947-4745-861a-9e9324abd263-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:58:20.716614 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.716588 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-kube-api-access-n5ffv" (OuterVolumeSpecName: "kube-api-access-n5ffv") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "kube-api-access-n5ffv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:58:20.716740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.716626 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11195ef4-b947-4745-861a-9e9324abd263-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 19:58:20.716740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.716632 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-istio-token" (OuterVolumeSpecName: "istio-token") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:58:20.716740 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.716636 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11195ef4-b947-4745-861a-9e9324abd263" (UID: "11195ef4-b947-4745-861a-9e9324abd263"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 19:58:20.816273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.816234 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-workload-certs\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.816273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.816264 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11195ef4-b947-4745-861a-9e9324abd263-istio-envoy\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.816273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.816273 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5ffv\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-kube-api-access-n5ffv\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.816273 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.816282 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11195ef4-b947-4745-861a-9e9324abd263-istio-podinfo\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.816546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.816290 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11195ef4-b947-4745-861a-9e9324abd263-istio-token\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:20.816546 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:20.816298 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11195ef4-b947-4745-861a-9e9324abd263-istiod-ca-cert\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:21.162939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.162856 2575 generic.go:356] "Generic (PLEG): container finished" podID="11195ef4-b947-4745-861a-9e9324abd263" containerID="b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73" exitCode=0 Sep 06 19:58:21.162939 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.162906 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" Sep 06 19:58:21.163117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.162945 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" event={"ID":"11195ef4-b947-4745-861a-9e9324abd263","Type":"ContainerDied","Data":"b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73"} Sep 06 19:58:21.163117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.163002 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx" event={"ID":"11195ef4-b947-4745-861a-9e9324abd263","Type":"ContainerDied","Data":"2695ab3084f8a56c8533ca470cdd2e0f9b3f5599e79af4eb7f9d0f4bb7879a59"} Sep 06 19:58:21.163117 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.163030 2575 scope.go:117] "RemoveContainer" containerID="b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73" Sep 06 19:58:21.174399 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.174379 2575 scope.go:117] "RemoveContainer" containerID="95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97" Sep 06 19:58:21.185603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.185476 2575 scope.go:117] "RemoveContainer" containerID="b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73" Sep 06 19:58:21.185603 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.185563 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx"] Sep 06 19:58:21.185971 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:58:21.185947 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73\": container with ID starting with b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73 not found: ID does not exist" containerID="b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73" Sep 06 19:58:21.186064 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.185982 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73"} err="failed to get container status \"b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73\": rpc error: code = NotFound desc = could not find container \"b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73\": container with ID starting with b64f55eed8c7afad53010da40c97597b98ddbcedfe25f17db2c0abcab5692c73 not found: ID does not exist" Sep 06 19:58:21.186064 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.186007 2575 scope.go:117] "RemoveContainer" containerID="95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97" Sep 06 19:58:21.186333 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:58:21.186296 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97\": container with ID starting with 95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97 not found: ID does not exist" containerID="95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97" Sep 06 19:58:21.186432 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.186332 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97"} err="failed to get container status \"95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97\": rpc error: code = NotFound desc = could not find container \"95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97\": container with ID starting with 95e3055e71c5b897ce747bb812f34bf35e03ddb386a2fbf32b647bd9a0935c97 not found: ID does not exist" Sep 06 19:58:21.188154 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.188130 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b571c4-openshift-default-77bcc7845b-rn9qx"] Sep 06 19:58:21.762202 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:21.762170 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11195ef4-b947-4745-861a-9e9324abd263" path="/var/lib/kubelet/pods/11195ef4-b947-4745-861a-9e9324abd263/volumes" Sep 06 19:58:26.183635 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.183595 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q"] Sep 06 19:58:26.184142 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.183864 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" podUID="a1a93431-9d77-4963-b729-887b5c8ab11c" containerName="payload-processing" containerID="cri-o://6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe" gracePeriod=30 Sep 06 19:58:26.190012 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.189979 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd"] Sep 06 19:58:26.190235 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.190201 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" podUID="85d4e583-3537-4daa-baf8-d7df7c6c4449" containerName="payload-pre-processing" containerID="cri-o://4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765" gracePeriod=30 Sep 06 19:58:26.464669 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.464645 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:58:26.467338 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.467320 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:58:26.573350 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.573317 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfp5l\" (UniqueName: \"kubernetes.io/projected/a1a93431-9d77-4963-b729-887b5c8ab11c-kube-api-access-tfp5l\") pod \"a1a93431-9d77-4963-b729-887b5c8ab11c\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " Sep 06 19:58:26.573350 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.573358 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d4e583-3537-4daa-baf8-d7df7c6c4449-plugins-config-volume\") pod \"85d4e583-3537-4daa-baf8-d7df7c6c4449\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " Sep 06 19:58:26.573579 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.573394 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w699f\" (UniqueName: \"kubernetes.io/projected/85d4e583-3537-4daa-baf8-d7df7c6c4449-kube-api-access-w699f\") pod \"85d4e583-3537-4daa-baf8-d7df7c6c4449\" (UID: \"85d4e583-3537-4daa-baf8-d7df7c6c4449\") " Sep 06 19:58:26.573579 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.573483 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a93431-9d77-4963-b729-887b5c8ab11c-plugins-config-volume\") pod \"a1a93431-9d77-4963-b729-887b5c8ab11c\" (UID: \"a1a93431-9d77-4963-b729-887b5c8ab11c\") " Sep 06 19:58:26.573911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.573876 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d4e583-3537-4daa-baf8-d7df7c6c4449-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85d4e583-3537-4daa-baf8-d7df7c6c4449" (UID: "85d4e583-3537-4daa-baf8-d7df7c6c4449"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:58:26.573911 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.573907 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a93431-9d77-4963-b729-887b5c8ab11c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1a93431-9d77-4963-b729-887b5c8ab11c" (UID: "a1a93431-9d77-4963-b729-887b5c8ab11c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 19:58:26.575736 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.575706 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a93431-9d77-4963-b729-887b5c8ab11c-kube-api-access-tfp5l" (OuterVolumeSpecName: "kube-api-access-tfp5l") pod "a1a93431-9d77-4963-b729-887b5c8ab11c" (UID: "a1a93431-9d77-4963-b729-887b5c8ab11c"). InnerVolumeSpecName "kube-api-access-tfp5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:58:26.575736 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.575713 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d4e583-3537-4daa-baf8-d7df7c6c4449-kube-api-access-w699f" (OuterVolumeSpecName: "kube-api-access-w699f") pod "85d4e583-3537-4daa-baf8-d7df7c6c4449" (UID: "85d4e583-3537-4daa-baf8-d7df7c6c4449"). InnerVolumeSpecName "kube-api-access-w699f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 19:58:26.674502 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.674468 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a93431-9d77-4963-b729-887b5c8ab11c-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:26.674502 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.674497 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfp5l\" (UniqueName: \"kubernetes.io/projected/a1a93431-9d77-4963-b729-887b5c8ab11c-kube-api-access-tfp5l\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:26.674502 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.674507 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d4e583-3537-4daa-baf8-d7df7c6c4449-plugins-config-volume\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:26.674739 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:26.674517 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w699f\" (UniqueName: \"kubernetes.io/projected/85d4e583-3537-4daa-baf8-d7df7c6c4449-kube-api-access-w699f\") on node \"ip-10-0-135-236.ec2.internal\" DevicePath \"\"" Sep 06 19:58:27.202859 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.202825 2575 generic.go:356] "Generic (PLEG): container finished" podID="a1a93431-9d77-4963-b729-887b5c8ab11c" containerID="6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe" exitCode=0 Sep 06 19:58:27.203271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.202877 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" Sep 06 19:58:27.203271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.202922 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" event={"ID":"a1a93431-9d77-4963-b729-887b5c8ab11c","Type":"ContainerDied","Data":"6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe"} Sep 06 19:58:27.203271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.202975 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q" event={"ID":"a1a93431-9d77-4963-b729-887b5c8ab11c","Type":"ContainerDied","Data":"7b4f770205515d246047b917f09dd2f7b4b2a8c835a66b2c3e6c3125435beb48"} Sep 06 19:58:27.203271 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.203005 2575 scope.go:117] "RemoveContainer" containerID="6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe" Sep 06 19:58:27.204587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.204370 2575 generic.go:356] "Generic (PLEG): container finished" podID="85d4e583-3537-4daa-baf8-d7df7c6c4449" containerID="4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765" exitCode=0 Sep 06 19:58:27.204587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.204405 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" event={"ID":"85d4e583-3537-4daa-baf8-d7df7c6c4449","Type":"ContainerDied","Data":"4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765"} Sep 06 19:58:27.204587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.204427 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" Sep 06 19:58:27.204587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.204461 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd" event={"ID":"85d4e583-3537-4daa-baf8-d7df7c6c4449","Type":"ContainerDied","Data":"8087b8e035a21d2a10d9be4d41d43a24e5ebfba68aaf3806e6dd72bfec3e19fe"} Sep 06 19:58:27.214206 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.214187 2575 scope.go:117] "RemoveContainer" containerID="6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe" Sep 06 19:58:27.214500 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:58:27.214481 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe\": container with ID starting with 6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe not found: ID does not exist" containerID="6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe" Sep 06 19:58:27.214587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.214509 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe"} err="failed to get container status \"6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe\": rpc error: code = NotFound desc = could not find container \"6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe\": container with ID starting with 6f47bf25ff26045830a5bb0ed46c747a93f1556fe9498414ef939367aaac58fe not found: ID does not exist" Sep 06 19:58:27.214587 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.214532 2575 scope.go:117] "RemoveContainer" containerID="4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765" Sep 06 19:58:27.223130 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.223085 2575 scope.go:117] "RemoveContainer" containerID="4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765" Sep 06 19:58:27.223400 ip-10-0-135-236 kubenswrapper[2575]: E0906 19:58:27.223382 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765\": container with ID starting with 4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765 not found: ID does not exist" containerID="4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765" Sep 06 19:58:27.223460 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.223407 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765"} err="failed to get container status \"4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765\": rpc error: code = NotFound desc = could not find container \"4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765\": container with ID starting with 4f2a4fde4726cb3adcd070c165630b9e89e555edf30d5722003155637b092765 not found: ID does not exist" Sep 06 19:58:27.225893 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.225875 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q"] Sep 06 19:58:27.230540 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.230520 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-712616-5dfb8ff89f-cql5q"] Sep 06 19:58:27.240203 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.240176 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd"] Sep 06 19:58:27.245642 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.245578 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-712616-758bd7d846-gkgrd"] Sep 06 19:58:27.764087 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.764052 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85d4e583-3537-4daa-baf8-d7df7c6c4449" path="/var/lib/kubelet/pods/85d4e583-3537-4daa-baf8-d7df7c6c4449/volumes" Sep 06 19:58:27.764418 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:58:27.764405 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a93431-9d77-4963-b729-887b5c8ab11c" path="/var/lib/kubelet/pods/a1a93431-9d77-4963-b729-887b5c8ab11c/volumes" Sep 06 19:59:38.125580 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:59:38.125546 2575 scope.go:117] "RemoveContainer" containerID="b1eec8c1bfe448395d41e56f1fe85d4861834542781bbdb393f45aa97f1b883d" Sep 06 19:59:38.134520 ip-10-0-135-236 kubenswrapper[2575]: I0906 19:59:38.134502 2575 scope.go:117] "RemoveContainer" containerID="e0bc81e0355d36fd9e0ffd252ecdbcdd74fcfa96f341f6e7316558900a3e5bae" Sep 06 20:00:00.129878 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.129836 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm"] Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130178 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85d4e583-3537-4daa-baf8-d7df7c6c4449" containerName="payload-pre-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130189 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="85d4e583-3537-4daa-baf8-d7df7c6c4449" containerName="payload-pre-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130200 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130206 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130217 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1296006-07b9-4608-b8a1-839eb0be31dc" containerName="revoke-keys" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130224 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1296006-07b9-4608-b8a1-839eb0be31dc" containerName="revoke-keys" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130230 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1507c2-d633-4519-845e-72986af14d2d" containerName="payload-pre-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130236 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1507c2-d633-4519-845e-72986af14d2d" containerName="payload-pre-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130247 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130251 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130258 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a93431-9d77-4963-b729-887b5c8ab11c" containerName="payload-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130263 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a93431-9d77-4963-b729-887b5c8ab11c" containerName="payload-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130271 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d395a0-c72b-4264-bfaf-c7ab376768e8" containerName="payload-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130276 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d395a0-c72b-4264-bfaf-c7ab376768e8" containerName="payload-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130284 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130291 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130361 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130370 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a93431-9d77-4963-b729-887b5c8ab11c" containerName="payload-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130376 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130383 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130390 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="85d4e583-3537-4daa-baf8-d7df7c6c4449" containerName="payload-pre-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130396 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1507c2-d633-4519-845e-72986af14d2d" containerName="payload-pre-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130401 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d395a0-c72b-4264-bfaf-c7ab376768e8" containerName="payload-processing" Sep 06 20:00:00.130395 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.130407 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1296006-07b9-4608-b8a1-839eb0be31dc" containerName="revoke-keys" Sep 06 20:00:00.133531 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.133515 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" Sep 06 20:00:00.136308 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.136270 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h45ww\"" Sep 06 20:00:00.138454 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.138426 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm"] Sep 06 20:00:00.145674 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.145653 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" Sep 06 20:00:00.424103 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.424067 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm"] Sep 06 20:00:00.427654 ip-10-0-135-236 kubenswrapper[2575]: W0906 20:00:00.427629 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4cb552_3837_4a37_81ec_2e84337d2288.slice/crio-dfcdd981b23ea04fddb7d87719289ce69c046916be90f487d86777f9a4da019e WatchSource:0}: Error finding container dfcdd981b23ea04fddb7d87719289ce69c046916be90f487d86777f9a4da019e: Status 404 returned error can't find the container with id dfcdd981b23ea04fddb7d87719289ce69c046916be90f487d86777f9a4da019e Sep 06 20:00:00.729932 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.729840 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" event={"ID":"fc4cb552-3837-4a37-81ec-2e84337d2288","Type":"ContainerStarted","Data":"dc5abf84f51803e7baf677923b3f23b9b87442ec77b76ffcd24abc3ffb8f3192"} Sep 06 20:00:00.729932 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.729899 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" event={"ID":"fc4cb552-3837-4a37-81ec-2e84337d2288","Type":"ContainerStarted","Data":"dfcdd981b23ea04fddb7d87719289ce69c046916be90f487d86777f9a4da019e"} Sep 06 20:00:00.745610 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:00.745561 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" podStartSLOduration=0.745545711 podStartE2EDuration="745.545711ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 20:00:00.742275901 +0000 UTC m=+1943.476880634" watchObservedRunningTime="2026-09-06 20:00:00.745545711 +0000 UTC m=+1943.480150442" Sep 06 20:00:01.737333 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:01.737303 2575 generic.go:356] "Generic (PLEG): container finished" podID="fc4cb552-3837-4a37-81ec-2e84337d2288" containerID="dc5abf84f51803e7baf677923b3f23b9b87442ec77b76ffcd24abc3ffb8f3192" exitCode=0 Sep 06 20:00:01.737772 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:01.737390 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" event={"ID":"fc4cb552-3837-4a37-81ec-2e84337d2288","Type":"ContainerDied","Data":"dc5abf84f51803e7baf677923b3f23b9b87442ec77b76ffcd24abc3ffb8f3192"} Sep 06 20:00:02.873036 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:02.873013 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" Sep 06 20:00:03.749666 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:03.749638 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" Sep 06 20:00:03.749852 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:03.749666 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29812080-d6kzm" event={"ID":"fc4cb552-3837-4a37-81ec-2e84337d2288","Type":"ContainerDied","Data":"dfcdd981b23ea04fddb7d87719289ce69c046916be90f487d86777f9a4da019e"} Sep 06 20:00:03.749852 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:03.749742 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfcdd981b23ea04fddb7d87719289ce69c046916be90f487d86777f9a4da019e" Sep 06 20:00:38.158536 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:38.158452 2575 scope.go:117] "RemoveContainer" containerID="e22ad390884c3e4d0d2d940026bdd678b07461ed2ee0edc9b7e43ccbfa62be24" Sep 06 20:00:38.167496 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:00:38.167474 2575 scope.go:117] "RemoveContainer" containerID="8d66f4df272a1504c1e764deffc5d55883b60408c3ab419b6d81611bd0a98fe6" Sep 06 20:02:37.878331 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:02:37.878303 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 20:02:37.884497 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:02:37.884477 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqg62_4b4ba88f-bbf9-4cd6-8be7-03a2c27091f3/ovn-acl-logging/0.log" Sep 06 20:02:38.200176 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:02:38.200145 2575 scope.go:117] "RemoveContainer" containerID="bfa5fe9e480028d1a91a179a497c4ebc8c3847938b66f32618717a8e121f23ab" Sep 06 20:02:38.209122 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:02:38.209101 2575 scope.go:117] "RemoveContainer" containerID="056476d18576dddcc130bc516b6bdf763903bae740e5b85611c1db39b2ed78ca" Sep 06 20:04:16.577611 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:16.577577 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4gtxj_742cb1c3-7009-4642-bc2c-f99378b5d102/server/0.log" Sep 06 20:04:16.699588 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:16.699555 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hrrwn_e6cee458-63cf-48ee-a6b4-ee109ef946ed/manager/0.log" Sep 06 20:04:16.803617 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:16.803586 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f964dcc99-2fbcr_4be8c382-9dd0-4314-b57e-fe0997aa05eb/copy-manifests/0.log" Sep 06 20:04:16.809859 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:16.809833 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f964dcc99-2fbcr_4be8c382-9dd0-4314-b57e-fe0997aa05eb/manager/0.log" Sep 06 20:04:17.698035 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:17.698005 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29812080-d6kzm_fc4cb552-3837-4a37-81ec-2e84337d2288/cleanup/0.log" Sep 06 20:04:19.540357 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:19.540321 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2bpth_693305e5-c0c7-4161-a927-f0858b17e555/istio-proxy/0.log" Sep 06 20:04:19.655866 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:19.655830 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-w2z8v_53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d/discovery/0.log" Sep 06 20:04:20.224539 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:20.224502 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dc6h2_881dd41f-b5f1-4930-8e55-a219f83580f1/payload-processing/0.log" Sep 06 20:04:20.331934 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:20.331891 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bc94b9958-gc6br_34fe05ee-374e-477f-96b3-df13f2182b01/router/0.log" Sep 06 20:04:20.764151 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:20.764113 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mvc6z_ac48f8e5-b322-4c8d-848e-027bf79cbdbc/main/0.log" Sep 06 20:04:21.203584 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:21.203557 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9jzp7_c2278452-df71-4eaa-bf5c-162e8f5e2025/main/0.log" Sep 06 20:04:29.974683 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:29.974651 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4f985_4c4447ca-1b3a-4eca-a32d-ea055bb5cbb4/global-pull-secret-syncer/0.log" Sep 06 20:04:30.132244 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:30.132208 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qnljk_ae7b9294-acfa-4b8d-88af-91860510096d/konnectivity-agent/0.log" Sep 06 20:04:30.207694 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:30.207662 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-236.ec2.internal_72e1c97d5f33b9f41328e5f5e35a0ce0/haproxy/0.log" Sep 06 20:04:37.536964 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.536922 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9k76n_7e38bc82-d6b2-4338-9568-a5b4a515afa1/kube-state-metrics/0.log" Sep 06 20:04:37.553769 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.553677 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9k76n_7e38bc82-d6b2-4338-9568-a5b4a515afa1/kube-rbac-proxy-main/0.log" Sep 06 20:04:37.575027 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.575002 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9k76n_7e38bc82-d6b2-4338-9568-a5b4a515afa1/kube-rbac-proxy-self/0.log" Sep 06 20:04:37.642382 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.642352 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-6688b_73c87f71-a35f-4bcc-b91e-9aaa6576e33b/monitoring-plugin/0.log" Sep 06 20:04:37.738509 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.738472 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g6qgw_71adaefa-a7e3-47a9-936a-5351a87a42be/node-exporter/0.log" Sep 06 20:04:37.756071 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.756044 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g6qgw_71adaefa-a7e3-47a9-936a-5351a87a42be/kube-rbac-proxy/0.log" Sep 06 20:04:37.773737 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.773712 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g6qgw_71adaefa-a7e3-47a9-936a-5351a87a42be/init-textfile/0.log" Sep 06 20:04:37.864192 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.864105 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zh44m_f40225d8-c52b-45cc-9140-002f4aaa4c05/kube-rbac-proxy-main/0.log" Sep 06 20:04:37.884172 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.884141 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zh44m_f40225d8-c52b-45cc-9140-002f4aaa4c05/kube-rbac-proxy-self/0.log" Sep 06 20:04:37.905534 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:37.905494 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zh44m_f40225d8-c52b-45cc-9140-002f4aaa4c05/openshift-state-metrics/0.log" Sep 06 20:04:38.243120 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:38.243088 2575 scope.go:117] "RemoveContainer" containerID="c28312f68d4e8f1a7e7c05102497dccb0df1335b7c1c4fea84e20f750c974566" Sep 06 20:04:39.656685 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:39.656646 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-x6nsc_d1bf5b94-eaf1-4e88-96b2-153f7a967e2a/networking-console-plugin/0.log" Sep 06 20:04:40.456788 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.456733 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v"] Sep 06 20:04:40.457122 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.457110 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" Sep 06 20:04:40.457172 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.457124 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="11195ef4-b947-4745-861a-9e9324abd263" containerName="istio-proxy" Sep 06 20:04:40.457172 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.457133 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4cb552-3837-4a37-81ec-2e84337d2288" containerName="cleanup" Sep 06 20:04:40.457172 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.457139 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4cb552-3837-4a37-81ec-2e84337d2288" containerName="cleanup" Sep 06 20:04:40.457267 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.457203 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b404a2a-e6f3-4833-87ec-97c7031f428e" containerName="istio-proxy" Sep 06 20:04:40.457267 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.457215 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4cb552-3837-4a37-81ec-2e84337d2288" containerName="cleanup" Sep 06 20:04:40.461692 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.461673 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.464100 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.464074 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4b6gr\"/\"kube-root-ca.crt\"" Sep 06 20:04:40.464927 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.464908 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4b6gr\"/\"openshift-service-ca.crt\"" Sep 06 20:04:40.465024 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.464908 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4b6gr\"/\"default-dockercfg-g8zxj\"" Sep 06 20:04:40.470436 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.470416 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v"] Sep 06 20:04:40.580639 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.580603 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-sys\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.580639 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.580641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-lib-modules\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.580906 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.580666 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-proc\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.580906 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.580792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbqbj\" (UniqueName: \"kubernetes.io/projected/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-kube-api-access-fbqbj\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.580906 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.580845 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-podres\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681451 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681415 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbqbj\" (UniqueName: \"kubernetes.io/projected/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-kube-api-access-fbqbj\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-podres\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681567 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-sys\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681606 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-lib-modules\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681648 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-proc\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681684 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-sys\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681684 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-podres\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681743 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-proc\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.681853 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.681793 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-lib-modules\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.700211 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.700177 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbqbj\" (UniqueName: \"kubernetes.io/projected/b4e9da50-5eeb-475d-9cf5-692e1d2d73fa-kube-api-access-fbqbj\") pod \"perf-node-gather-daemonset-fzn5v\" (UID: \"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa\") " pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:40.771329 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.771252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bc997b6c8-mz6b6_0d5d9b7c-299d-4dcb-b02e-de9e90ab0ce3/console/0.log" Sep 06 20:04:40.772968 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:40.772943 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:41.045214 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:41.045129 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v"] Sep 06 20:04:41.048445 ip-10-0-135-236 kubenswrapper[2575]: W0906 20:04:41.048414 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb4e9da50_5eeb_475d_9cf5_692e1d2d73fa.slice/crio-1c7a2e49785b8bc383b76d7a2ba337ed7757e161e1113b19532ce83b961c6cba WatchSource:0}: Error finding container 1c7a2e49785b8bc383b76d7a2ba337ed7757e161e1113b19532ce83b961c6cba: Status 404 returned error can't find the container with id 1c7a2e49785b8bc383b76d7a2ba337ed7757e161e1113b19532ce83b961c6cba Sep 06 20:04:41.253985 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:41.253948 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" event={"ID":"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa","Type":"ContainerStarted","Data":"80f4a7479d3a572f46a0ad952332b1b702102e24fa7b708d32b5d27df9c74cfc"} Sep 06 20:04:41.254161 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:41.254000 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" event={"ID":"b4e9da50-5eeb-475d-9cf5-692e1d2d73fa","Type":"ContainerStarted","Data":"1c7a2e49785b8bc383b76d7a2ba337ed7757e161e1113b19532ce83b961c6cba"} Sep 06 20:04:41.254161 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:41.254028 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:41.269010 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:41.268961 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" podStartSLOduration=1.268946491 podStartE2EDuration="1.268946491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 20:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 20:04:41.266988473 +0000 UTC m=+2224.001593207" watchObservedRunningTime="2026-09-06 20:04:41.268946491 +0000 UTC m=+2224.003551222" Sep 06 20:04:42.558143 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:42.558113 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-48bqg_cb875ad2-e098-4759-b872-1ed9f3e2fabd/dns/0.log" Sep 06 20:04:42.574849 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:42.574815 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-48bqg_cb875ad2-e098-4759-b872-1ed9f3e2fabd/kube-rbac-proxy/0.log" Sep 06 20:04:42.689263 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:42.689240 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jbbtr_052d0c66-9223-419b-980f-e592d8e633f0/dns-node-resolver/0.log" Sep 06 20:04:43.232825 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:43.232790 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5c65f4f9b-whcnh_8643a868-3d64-4e56-8780-305c9d7eeb60/registry/0.log" Sep 06 20:04:43.274317 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:43.274268 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-th522_24a08ec0-deb6-41e9-93ff-8234c06fd1c4/node-ca/0.log" Sep 06 20:04:44.185601 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:44.185572 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2bpth_693305e5-c0c7-4161-a927-f0858b17e555/istio-proxy/0.log" Sep 06 20:04:44.751909 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:44.751870 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-w2z8v_53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d/discovery/0.log" Sep 06 20:04:44.926216 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:44.926184 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dc6h2_881dd41f-b5f1-4930-8e55-a219f83580f1/payload-processing/0.log" Sep 06 20:04:44.945664 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:44.945631 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bc94b9958-gc6br_34fe05ee-374e-477f-96b3-df13f2182b01/router/0.log" Sep 06 20:04:45.831984 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:45.831935 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v49bv_fd1bce41-2076-4298-879d-14c5ac56a61c/serve-healthcheck-canary/0.log" Sep 06 20:04:46.278543 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:46.278507 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dblnk_d15c8c28-2067-439e-b876-0575479656de/insights-operator/0.log" Sep 06 20:04:46.278839 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:46.278805 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dblnk_d15c8c28-2067-439e-b876-0575479656de/insights-operator/1.log" Sep 06 20:04:46.363216 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:46.363184 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l6499_99ec35e2-484c-420d-82b0-39b1f63909be/kube-rbac-proxy/0.log" Sep 06 20:04:46.379404 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:46.379379 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l6499_99ec35e2-484c-420d-82b0-39b1f63909be/exporter/0.log" Sep 06 20:04:46.396146 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:46.396120 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l6499_99ec35e2-484c-420d-82b0-39b1f63909be/extractor/0.log" Sep 06 20:04:47.269813 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:47.269786 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4b6gr/perf-node-gather-daemonset-fzn5v" Sep 06 20:04:48.902127 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:48.902094 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4gtxj_742cb1c3-7009-4642-bc2c-f99378b5d102/server/0.log" Sep 06 20:04:49.021898 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:49.021864 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hrrwn_e6cee458-63cf-48ee-a6b4-ee109ef946ed/manager/0.log" Sep 06 20:04:49.045595 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:49.045566 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f964dcc99-2fbcr_4be8c382-9dd0-4314-b57e-fe0997aa05eb/manager/0.log" Sep 06 20:04:49.061926 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:49.061909 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f964dcc99-2fbcr_4be8c382-9dd0-4314-b57e-fe0997aa05eb/copy-manifests/0.log" Sep 06 20:04:50.689875 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:50.689843 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7769675c48-b565m_0e34c8f9-5d4d-41cd-9bce-435e6cbf937e/manager/0.log" Sep 06 20:04:51.350481 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:51.350435 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2bpth_693305e5-c0c7-4161-a927-f0858b17e555/istio-proxy/0.log" Sep 06 20:04:51.913882 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:51.913797 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-w2z8v_53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d/discovery/0.log" Sep 06 20:04:52.055061 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:52.055023 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dc6h2_881dd41f-b5f1-4930-8e55-a219f83580f1/payload-processing/0.log" Sep 06 20:04:52.069944 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:52.069908 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bc94b9958-gc6br_34fe05ee-374e-477f-96b3-df13f2182b01/router/0.log" Sep 06 20:04:53.536586 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:53.536539 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2bpth_693305e5-c0c7-4161-a927-f0858b17e555/istio-proxy/0.log" Sep 06 20:04:54.127372 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:54.127337 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-w2z8v_53e3dc65-c3d6-4f3c-a71b-6c984cfe6c5d/discovery/0.log" Sep 06 20:04:54.265033 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:54.265002 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dc6h2_881dd41f-b5f1-4930-8e55-a219f83580f1/payload-processing/0.log" Sep 06 20:04:54.282618 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:54.282589 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6bc94b9958-gc6br_34fe05ee-374e-477f-96b3-df13f2182b01/router/0.log" Sep 06 20:04:55.974059 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:55.974025 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-d7hpj_09828082-8a90-466f-85ff-02776367a618/kube-storage-version-migrator-operator/1.log" Sep 06 20:04:55.975691 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:55.975669 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-d7hpj_09828082-8a90-466f-85ff-02776367a618/kube-storage-version-migrator-operator/0.log" Sep 06 20:04:56.856936 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.856905 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/kube-multus-additional-cni-plugins/0.log" Sep 06 20:04:56.875636 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.875605 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/egress-router-binary-copy/0.log" Sep 06 20:04:56.893964 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.893935 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/cni-plugins/0.log" Sep 06 20:04:56.910026 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.910004 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/bond-cni-plugin/0.log" Sep 06 20:04:56.926694 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.926676 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/routeoverride-cni/0.log" Sep 06 20:04:56.943340 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.943310 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/whereabouts-cni-bincopy/0.log" Sep 06 20:04:56.959965 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:56.959938 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hmmz8_16b7f01c-8c0b-4b1d-944e-9ca9418371b3/whereabouts-cni/0.log" Sep 06 20:04:57.368708 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:57.368675 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v7nd9_349c8685-ef1a-4e22-b5d4-20fc72aac46a/kube-multus/0.log" Sep 06 20:04:57.435220 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:57.435188 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p2qnt_46414df0-60db-46f7-91d2-5378dff237aa/network-metrics-daemon/0.log" Sep 06 20:04:57.449496 ip-10-0-135-236 kubenswrapper[2575]: I0906 20:04:57.449472 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p2qnt_46414df0-60db-46f7-91d2-5378dff237aa/kube-rbac-proxy/0.log"