Sep 10 19:52:12.880852 ip-10-0-130-59 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 19:52:12.880981 ip-10-0-130-59 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 19:52:12.881065 ip-10-0-130-59 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 19:52:12.881559 ip-10-0-130-59 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 19:52:22.310288 ip-10-0-130-59 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 19:52:22.310303 ip-10-0-130-59 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f6483102dc3b49a0b21cdb33e9a42923 -- Sep 10 19:53:51.148429 ip-10-0-130-59 systemd[1]: Starting Kubernetes Kubelet... Sep 10 19:53:51.597958 ip-10-0-130-59 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 19:53:51.597958 ip-10-0-130-59 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 19:53:51.597958 ip-10-0-130-59 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 19:53:51.597958 ip-10-0-130-59 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 19:53:51.597958 ip-10-0-130-59 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 19:53:51.599715 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.599627 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 19:53:51.605907 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605890 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 19:53:51.605907 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605907 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605911 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605915 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605918 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605921 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605924 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605927 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605930 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605933 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605935 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605938 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605940 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605943 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605946 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605948 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605951 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605953 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605956 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605959 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605961 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 19:53:51.605972 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605964 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605972 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605975 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605978 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605981 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605984 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605986 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605989 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605993 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.605997 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606000 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606003 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606006 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606009 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606011 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606014 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606016 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606019 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606021 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606024 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 19:53:51.606470 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606026 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606029 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606031 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606040 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606044 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606047 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606050 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606053 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606055 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606058 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606061 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606064 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606077 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606081 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606084 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606087 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606090 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606093 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606096 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 19:53:51.606990 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606098 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606101 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606103 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606106 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606108 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606111 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606113 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606116 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606118 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606121 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606123 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606126 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606128 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606131 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606133 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606136 2570 feature_gate.go:328] unrecognized feature gate: Example Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606139 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606147 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606150 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 19:53:51.607471 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606153 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606157 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606161 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606164 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606167 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606170 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606172 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606175 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606179 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606181 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606184 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606187 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606190 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606193 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606195 2570 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606198 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606200 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606203 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606205 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606208 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 19:53:51.607942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.606999 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607006 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607010 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607012 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607015 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607018 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607021 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607024 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607027 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607030 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607032 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607041 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607043 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607046 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607049 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607051 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607054 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607057 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607060 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607063 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 19:53:51.608443 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607066 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607082 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607084 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607087 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607090 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607092 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607095 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607098 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607100 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607102 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607105 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607107 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607110 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607112 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607115 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607117 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607120 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607122 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607125 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 19:53:51.608938 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607127 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607130 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607132 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607135 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607137 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607149 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607152 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607154 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607157 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607162 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607164 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607168 2570 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607170 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607173 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607175 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607178 2570 feature_gate.go:328] unrecognized feature gate: Example Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607180 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607183 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607186 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607188 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 19:53:51.609421 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607191 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607193 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607196 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607198 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607201 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607203 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607206 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607209 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607212 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607214 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607218 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607222 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607226 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607229 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607232 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607235 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607237 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607240 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607249 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 19:53:51.609912 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607251 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607254 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607258 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607260 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607263 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607265 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607268 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607270 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607273 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607276 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607278 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607282 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607284 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607287 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607290 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607292 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607295 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607298 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607302 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 19:53:51.610385 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607305 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607307 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607718 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607725 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607729 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607732 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607735 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607737 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607740 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607743 2570 feature_gate.go:328] unrecognized feature gate: Example Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607747 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607749 2570 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607753 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607756 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607758 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607763 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607765 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607768 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607771 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607773 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 19:53:51.610866 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607776 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607778 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607781 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607784 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607786 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607789 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607791 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607794 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607796 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607799 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607802 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607804 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607807 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607809 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607812 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607814 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607817 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607819 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607822 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 19:53:51.611366 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607825 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607829 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607832 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607835 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607838 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607842 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607845 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607848 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607852 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607855 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607858 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607860 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607863 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607866 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607869 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607871 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607874 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607876 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607880 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 19:53:51.611835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607882 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607885 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607887 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607890 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607892 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607895 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607897 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607902 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607905 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607908 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607911 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607914 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607916 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607919 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607922 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607924 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607927 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607929 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607931 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607934 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 19:53:51.612353 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607937 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607941 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607943 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607946 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607949 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607951 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607954 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607957 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607959 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607962 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607965 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607967 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607970 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607972 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607974 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607977 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607980 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607982 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607985 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607987 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 19:53:51.612845 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607990 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607992 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.607996 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608084 2570 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608125 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608134 2570 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608139 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608144 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608147 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608151 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608156 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608160 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608162 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608168 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608171 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608174 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608178 2570 flags.go:64] FLAG: --cgroup-root="" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608181 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608184 2570 flags.go:64] FLAG: --client-ca-file="" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608187 2570 flags.go:64] FLAG: --cloud-provider="external" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608190 2570 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608194 2570 flags.go:64] FLAG: --cluster-domain="" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608197 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 19:53:51.613347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608200 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608203 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608206 2570 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608210 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608213 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608216 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608219 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608222 2570 flags.go:64] FLAG: --contention-profiling="false" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608225 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608228 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608231 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608235 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608239 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608242 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608245 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608248 2570 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608251 2570 flags.go:64] FLAG: --enable-server="true" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608254 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608262 2570 flags.go:64] FLAG: --event-burst="100" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608266 2570 flags.go:64] FLAG: --event-qps="50" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608269 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608272 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608278 2570 flags.go:64] FLAG: --eviction-hard="" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608282 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 19:53:51.613935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608285 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608288 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608291 2570 flags.go:64] FLAG: --eviction-soft="" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608294 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608297 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608300 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608303 2570 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608305 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608308 2570 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608311 2570 flags.go:64] FLAG: --feature-gates="" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608314 2570 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608318 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608321 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608324 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608327 2570 flags.go:64] FLAG: --healthz-port="10248" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608330 2570 flags.go:64] FLAG: --help="false" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608333 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608336 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608339 2570 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608343 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608347 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608350 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608353 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 19:53:51.614547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608356 2570 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608359 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608362 2570 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608365 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608368 2570 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608371 2570 flags.go:64] FLAG: --kube-reserved="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608374 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608377 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608381 2570 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608384 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608387 2570 flags.go:64] FLAG: --lock-file="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608390 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608393 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608396 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608401 2570 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608404 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608407 2570 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608410 2570 flags.go:64] FLAG: --logging-format="text" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608413 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608416 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608420 2570 flags.go:64] FLAG: --manifest-url="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608423 2570 flags.go:64] FLAG: --manifest-url-header="" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608427 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608430 2570 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608434 2570 flags.go:64] FLAG: --max-pods="110" Sep 10 19:53:51.615143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608437 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608440 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608443 2570 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608448 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608451 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608454 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608457 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608465 2570 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608468 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608471 2570 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608474 2570 flags.go:64] FLAG: --pod-cidr="" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608477 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608483 2570 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608486 2570 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608489 2570 flags.go:64] FLAG: --pods-per-core="0" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608493 2570 flags.go:64] FLAG: --port="10250" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608496 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608500 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08871785aff230f6d" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608504 2570 flags.go:64] FLAG: --qos-reserved="" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608507 2570 flags.go:64] FLAG: --read-only-port="10255" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608510 2570 flags.go:64] FLAG: --register-node="true" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608513 2570 flags.go:64] FLAG: --register-with-taints="" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608516 2570 flags.go:64] FLAG: --registry-burst="10" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608519 2570 flags.go:64] FLAG: --registry-qps="5" Sep 10 19:53:51.615763 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608522 2570 flags.go:64] FLAG: --reserved-cpus="" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608525 2570 flags.go:64] FLAG: --reserved-memory="" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608528 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608532 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608535 2570 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608538 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608541 2570 flags.go:64] FLAG: --runonce="false" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608544 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608547 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608550 2570 flags.go:64] FLAG: --seccomp-default="false" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608553 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608557 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608560 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608563 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608566 2570 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608569 2570 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608572 2570 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608575 2570 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608577 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608580 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608583 2570 flags.go:64] FLAG: --system-cgroups="" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608586 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608592 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608596 2570 flags.go:64] FLAG: --tls-cert-file="" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608599 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 19:53:51.616369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608602 2570 flags.go:64] FLAG: --tls-min-version="" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608606 2570 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608609 2570 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608612 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608615 2570 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608618 2570 flags.go:64] FLAG: --v="2" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608622 2570 flags.go:64] FLAG: --version="false" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608626 2570 flags.go:64] FLAG: --vmodule="" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608630 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.608633 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608749 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608753 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608756 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608760 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608763 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608766 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608769 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608772 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608775 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608778 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608781 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608783 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 19:53:51.617009 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608786 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608788 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608791 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608794 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608797 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608799 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608802 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608805 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608809 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608811 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608814 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608817 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608820 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608823 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608826 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608829 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608832 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608834 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608837 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 19:53:51.617609 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608839 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608843 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608847 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608850 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608853 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608856 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608859 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608861 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608864 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608867 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608870 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608873 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608876 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608879 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608881 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608884 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608887 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608889 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 19:53:51.618127 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608893 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608897 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608900 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608904 2570 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608907 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608910 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608912 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608915 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608918 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608920 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608923 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608926 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608929 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608931 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608934 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608937 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608940 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608943 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608945 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608948 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 19:53:51.618571 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608951 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608954 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608956 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608959 2570 feature_gate.go:328] unrecognized feature gate: Example Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608962 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608964 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608967 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608970 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608974 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608978 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608982 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608985 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608988 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608991 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608993 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.608997 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609006 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609010 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609013 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609015 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 19:53:51.619049 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609018 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609021 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609024 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609028 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609031 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609034 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609037 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609040 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609044 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.609047 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.609644 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.617577 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.617591 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.617608 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 19:53:51.619558 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.617614 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617670 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617675 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617679 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617683 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617686 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617690 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617693 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617696 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617698 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617701 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617704 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617706 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617709 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617712 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617715 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617717 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617720 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617722 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617724 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 19:53:51.619987 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617727 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617729 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617732 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617735 2570 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617737 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617740 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617742 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617745 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617747 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617750 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617753 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617756 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617758 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617761 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617764 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617767 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617770 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617772 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617775 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617777 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 19:53:51.620490 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617780 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617782 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617785 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617787 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617790 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617794 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617797 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617800 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617803 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617806 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617809 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617812 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617815 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617817 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617820 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617823 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617826 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617828 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617831 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617833 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 19:53:51.621011 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617836 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617839 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617841 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617845 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617847 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617850 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617852 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617855 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617857 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617860 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617862 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617865 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617867 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617870 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617872 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617875 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617877 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617880 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617883 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617885 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 19:53:51.621576 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617888 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617890 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617893 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617895 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617897 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617900 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617903 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617905 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617908 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617910 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617913 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617915 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617919 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617923 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617926 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617929 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617932 2570 feature_gate.go:328] unrecognized feature gate: Example Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617934 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617937 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 19:53:51.622066 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.617940 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.617945 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618055 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618060 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618064 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618066 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618084 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618089 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618094 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618097 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618100 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618103 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618106 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618109 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618112 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 19:53:51.622544 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618115 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618117 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618120 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618123 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618126 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618130 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618133 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618137 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618140 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618143 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618146 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618150 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618153 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618156 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618159 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618163 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618166 2570 feature_gate.go:328] unrecognized feature gate: Example Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618169 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618171 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 19:53:51.622911 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618174 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618177 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618179 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618182 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618184 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618187 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618189 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618191 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618194 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618196 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618199 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618201 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618204 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618207 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618209 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618211 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618214 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618217 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618219 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618222 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 19:53:51.623384 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618224 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618227 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618229 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618232 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618234 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618237 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618239 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618242 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618244 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618247 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618250 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618252 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618255 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618257 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618260 2570 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618262 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618265 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618267 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618270 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 19:53:51.623857 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618272 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618275 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618277 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618280 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618282 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618285 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618287 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618289 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618292 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618295 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618298 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618300 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618303 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618306 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618310 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618313 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618315 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618318 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618320 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618323 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 19:53:51.624323 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618325 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618327 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618330 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618333 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618336 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618338 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618341 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:51.618343 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.618348 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.618484 2570 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.621023 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.622176 2570 server.go:1019] "Starting client certificate rotation" Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.622271 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 19:53:51.624803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.622311 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 19:53:51.625214 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.625201 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 19:53:51.626908 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.626876 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 19:53:51.641038 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.641012 2570 log.go:25] "Validated CRI v1 runtime API" Sep 10 19:53:51.644722 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.644706 2570 log.go:25] "Validated CRI v1 image API" Sep 10 19:53:51.645599 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.645576 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 19:53:51.649481 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.649459 2570 fs.go:135] Filesystem UUIDs: map[2c4cb127-9132-40e3-bf33-a1d41185d26c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d73c5da6-bf30-41a4-b2a1-67eb29a234bb:/dev/nvme0n1p3] Sep 10 19:53:51.649544 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.649479 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 19:53:51.654996 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.654874 2570 manager.go:216] Machine: {Timestamp:2026-09-10 19:53:51.653487223 +0000 UTC m=+0.374434354 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095925 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27acb228bf3cb6ee04422bc6a3e830 SystemUUID:ec27acb2-28bf-3cb6-ee04-422bc6a3e830 BootID:f6483102-dc3b-49a0-b21c-db33e9a42923 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:27:db:07:37:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:27:db:07:37:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:1c:ab:d3:b8:8a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 19:53:51.654996 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.654986 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 19:53:51.655141 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.655082 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 19:53:51.656793 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.656776 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 19:53:51.659253 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.659225 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 19:53:51.659400 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.659258 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-59.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 19:53:51.660058 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.660048 2570 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 19:53:51.660106 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.660060 2570 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 19:53:51.660106 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.660079 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 19:53:51.661496 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.661485 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 19:53:51.663053 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.663043 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 19:53:51.667154 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667142 2570 state_mem.go:36] "Initialized new in-memory state store" Sep 10 19:53:51.667270 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667261 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 19:53:51.667300 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667296 2570 kubelet.go:486] "Attempting to sync node with API server" Sep 10 19:53:51.667327 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667306 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 19:53:51.667327 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667321 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 19:53:51.667382 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667330 2570 kubelet.go:398] "Adding apiserver pod source" Sep 10 19:53:51.667382 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667339 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 19:53:51.667804 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667784 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 19:53:51.667850 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.667810 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 19:53:51.669985 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.669968 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 19:53:51.672847 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.672833 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 19:53:51.672907 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.672866 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 19:53:51.673031 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.673021 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 19:53:51.673126 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.673038 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 19:53:51.673126 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.673044 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 19:53:51.673126 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.673050 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 19:53:51.673126 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.673058 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 19:53:51.673126 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.673063 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 19:53:51.674380 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.674369 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 19:53:51.674380 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.674380 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 19:53:51.674447 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.674387 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 19:53:51.674447 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.674393 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 19:53:51.674447 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.674401 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 19:53:51.674447 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.674416 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 19:53:51.677124 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.677108 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 19:53:51.677167 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.677128 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 19:53:51.677945 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.677923 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 19:53:51.677998 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.677975 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 19:53:51.681618 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.681605 2570 server.go:1268] "Started kubelet" Sep 10 19:53:51.681731 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.681691 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 19:53:51.681790 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.681764 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 19:53:51.681856 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.681795 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 19:53:51.685512 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.685487 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 19:53:51.685880 ip-10-0-130-59 systemd[1]: Started Kubernetes Kubelet. Sep 10 19:53:51.687026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.686998 2570 server.go:310] "Adding debug handlers to kubelet server" Sep 10 19:53:51.687135 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.687033 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 19:53:51.691235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.691219 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-59.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 19:53:51.692417 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.691356 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d40db704ad4fc6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 19:53:51.681572806 +0000 UTC m=+0.402519917,LastTimestamp:2026-09-10 19:53:51.681572806 +0000 UTC m=+0.402519917,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}" Sep 10 19:53:51.692561 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.692548 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 19:53:51.692605 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.692570 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 19:53:51.692669 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.692652 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:51.693331 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693303 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 19:53:51.693418 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693371 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 19:53:51.693418 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693387 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 19:53:51.693530 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693483 2570 reconstruct.go:97] "Volume reconstruction finished" Sep 10 19:53:51.693530 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693492 2570 reconciler.go:29] "Reconciler: start to sync state" Sep 10 19:53:51.693530 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693518 2570 factory.go:55] Registering systemd factory Sep 10 19:53:51.693649 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693536 2570 factory.go:223] Registration of the systemd container factory successfully Sep 10 19:53:51.693795 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693778 2570 factory.go:153] Registering CRI-O factory Sep 10 19:53:51.693857 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693800 2570 factory.go:223] Registration of the crio container factory successfully Sep 10 19:53:51.693857 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693852 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 19:53:51.693948 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693870 2570 factory.go:103] Registering Raw factory Sep 10 19:53:51.693948 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.693887 2570 manager.go:1196] Started watching for new ooms in manager Sep 10 19:53:51.694316 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.694304 2570 manager.go:349] Starting recovery of all containers Sep 10 19:53:51.698191 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.698162 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 19:53:51.698346 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.698314 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 19:53:51.705556 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.705537 2570 manager.go:354] Recovery completed Sep 10 19:53:51.707163 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.707145 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5rjc5" Sep 10 19:53:51.710024 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.710008 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.711872 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.711857 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.711931 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.711881 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.711931 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.711892 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.712575 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.712557 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 19:53:51.712575 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.712570 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 19:53:51.712689 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.712587 2570 state_mem.go:36] "Initialized new in-memory state store" Sep 10 19:53:51.712689 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.712610 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.713520 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.713504 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.713587 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.713534 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.713587 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.713545 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.713825 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.713756 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-59.ec2.internal.18d40db7067ba382 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-59.ec2.internal,UID:ip-10-0-130-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-59.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-59.ec2.internal,},FirstTimestamp:2026-09-10 19:53:51.711871874 +0000 UTC m=+0.432818987,LastTimestamp:2026-09-10 19:53:51.711871874 +0000 UTC m=+0.432818987,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-59.ec2.internal,}" Sep 10 19:53:51.713981 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.713968 2570 policy_none.go:49] "None policy: Start" Sep 10 19:53:51.714014 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.713989 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 19:53:51.714014 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.714003 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 19:53:51.715229 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.715218 2570 policy_none.go:47] "Start" Sep 10 19:53:51.715689 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.715670 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5rjc5" Sep 10 19:53:51.743872 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.743849 2570 manager.go:337] "Starting Device Plugin manager" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.743917 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.743931 2570 server.go:87] "Starting device plugin registration server" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.744208 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.744219 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.744302 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.744376 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.744385 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.744964 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.745024 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.757736 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.758925 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.758966 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.758986 2570 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.759022 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 19:53:51.762581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.760952 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 19:53:51.845284 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.845237 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.846322 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.846305 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.846391 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.846340 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.846391 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.846352 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.846391 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.846374 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.855576 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.855511 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.855576 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.855535 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-59.ec2.internal\": node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:51.859625 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.859599 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"] Sep 10 19:53:51.859696 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.859663 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.860421 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.860403 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.860514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.860432 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.860514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.860445 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.862547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.862534 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.862670 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.862655 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.862716 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.862687 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.863222 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.863203 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.863314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.863205 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.863314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.863259 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.863314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.863269 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.863314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.863231 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.863460 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.863318 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.866455 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.866441 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.866530 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.866464 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 19:53:51.867039 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.867024 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Sep 10 19:53:51.867122 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.867051 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 19:53:51.867122 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.867065 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Sep 10 19:53:51.884007 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.883864 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:51.895035 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.895019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.895121 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.895085 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.895171 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.895124 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48adfbd41af55e45ba98e8777ebfc093-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"48adfbd41af55e45ba98e8777ebfc093\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.895654 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.895641 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.899441 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.899427 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.984589 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:51.984565 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:51.995338 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.995314 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.995426 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.995340 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.995426 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.995356 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48adfbd41af55e45ba98e8777ebfc093-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"48adfbd41af55e45ba98e8777ebfc093\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.995426 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.995399 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48adfbd41af55e45ba98e8777ebfc093-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"48adfbd41af55e45ba98e8777ebfc093\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.995426 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.995410 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:51.995426 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:51.995422 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/667ccdd66cb316e2e58a7c2fdcf49d11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"667ccdd66cb316e2e58a7c2fdcf49d11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:52.085551 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.085501 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.186239 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.186174 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.198574 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.198555 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:52.203990 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.203484 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Sep 10 19:53:52.286488 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.286448 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.387003 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.386961 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.487584 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.487526 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.588452 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.588407 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.621680 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.621652 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 19:53:52.622095 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.621811 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 19:53:52.689126 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.689085 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.717278 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.717243 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 19:48:52 +0000 UTC" deadline="2028-06-20 23:38:10.522143104 +0000 UTC" Sep 10 19:53:52.717278 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.717272 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15579h44m17.804874472s" Sep 10 19:53:52.789814 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.789779 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.889157 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:52.889113 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48adfbd41af55e45ba98e8777ebfc093.slice/crio-69e58a69f274ae04b1cf0f955510fc5e75ade5b619e02bc5a99b721a00ab475a WatchSource:0}: Error finding container 69e58a69f274ae04b1cf0f955510fc5e75ade5b619e02bc5a99b721a00ab475a: Status 404 returned error can't find the container with id 69e58a69f274ae04b1cf0f955510fc5e75ade5b619e02bc5a99b721a00ab475a Sep 10 19:53:52.890229 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.890209 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:52.893336 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:52.893315 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 19:53:52.895023 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:52.895003 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667ccdd66cb316e2e58a7c2fdcf49d11.slice/crio-39fcfb9a6b9d5d4590e004e31ee5437be88731391c33dae04c9f8b2dda55fd82 WatchSource:0}: Error finding container 39fcfb9a6b9d5d4590e004e31ee5437be88731391c33dae04c9f8b2dda55fd82: Status 404 returned error can't find the container with id 39fcfb9a6b9d5d4590e004e31ee5437be88731391c33dae04c9f8b2dda55fd82 Sep 10 19:53:52.991132 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:52.991059 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:53.089273 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.089187 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 19:53:53.092209 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.092191 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:53.194115 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.194059 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Sep 10 19:53:53.198847 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.198829 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 19:53:53.236623 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.236593 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 19:53:53.292936 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.292903 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Sep 10 19:53:53.308571 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.308546 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 19:53:53.309906 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.309891 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Sep 10 19:53:53.319783 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.319764 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 19:53:53.671225 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.671190 2570 apiserver.go:52] "Watching apiserver" Sep 10 19:53:53.679898 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.679870 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 19:53:53.684280 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.683886 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-4crnq","openshift-dns/node-resolver-tpt2k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal","openshift-multus/multus-bqpkk","openshift-multus/network-metrics-daemon-dhzfs","openshift-ovn-kubernetes/ovnkube-node-tszp7","kube-system/global-pull-secret-syncer-flfcj","kube-system/konnectivity-agent-gxpgv","kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz","openshift-image-registry/node-ca-dspv5","openshift-multus/multus-additional-cni-plugins-jdtfj","openshift-network-diagnostics/network-check-target-2lgfz","openshift-network-operator/iptables-alerter-k4qdv"] Sep 10 19:53:53.686709 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.686683 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.686828 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.686778 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:53:53.689078 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.688965 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:53.689078 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.689038 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:53:53.691699 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.691301 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.693135 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.693039 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 19:53:53.693822 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.693802 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.694823 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.694800 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.695050 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.695028 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.695301 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.695282 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bn7jv\"" Sep 10 19:53:53.695515 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.695495 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 19:53:53.696202 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.696179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:53.696309 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.696253 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:53:53.696770 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.696744 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.697215 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.697195 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 19:53:53.697649 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.697451 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 19:53:53.697649 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.697560 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.697649 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.697620 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2c8q7\"" Sep 10 19:53:53.698555 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.698532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.701027 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.701004 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 19:53:53.701937 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.701918 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.702593 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.702572 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 19:53:53.702687 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.702677 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 19:53:53.702881 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.702861 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 19:53:53.702966 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.702885 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.703159 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.703137 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p8vsm\"" Sep 10 19:53:53.703393 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.703180 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.703783 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.703759 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2cd9622-ed11-4e4c-b30a-7d94dc097090-kubelet-config\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.703867 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.703856 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-socket-dir-parent\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.703931 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.703885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-k8s-cni-cncf-io\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.705052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.705014 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 19:53:53.705262 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.705246 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dkrpn\"" Sep 10 19:53:53.705492 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.705470 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 19:53:53.706314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.706293 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.707745 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707685 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-netns\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.707745 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-conf-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.707895 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-multus-certs\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.707895 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2cd9622-ed11-4e4c-b30a-7d94dc097090-dbus\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.707895 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707799 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-system-cni-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.707895 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-cni-multus\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.707895 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707849 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfx82\" (UniqueName: \"kubernetes.io/projected/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-kube-api-access-tfx82\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.707895 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707876 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707902 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-iptables-alerter-script\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76krq\" (UniqueName: \"kubernetes.io/projected/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-kube-api-access-76krq\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-cni-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.707993 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-cnibin\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-cni-binary-copy\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-cni-bin\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708077 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-hostroot\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708099 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708121 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-host-slash\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-kubelet\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708190 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-daemon-config\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708215 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-etc-kubernetes\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.708827 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708693 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.708932 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708862 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.708988 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708869 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mcln4\"" Sep 10 19:53:53.709083 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.708913 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 19:53:53.709597 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.709572 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 19:53:53.710348 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.710327 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.712813 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.712771 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.713752 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.713730 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.713879 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.713861 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xz2zl\"" Sep 10 19:53:53.714137 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.714118 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.715117 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.715096 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.716519 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.716497 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x8bzk\"" Sep 10 19:53:53.716614 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.716505 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.716787 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.716770 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.716957 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.716940 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 19:53:53.717392 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.717371 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 19:53:53.717652 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.717632 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 19:53:53.717753 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.717730 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.717896 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.717879 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 19:53:53.718127 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.718108 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tjpk5\"" Sep 10 19:53:53.720484 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.720462 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 19:53:53.720792 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.720770 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sr9sk\"" Sep 10 19:53:53.720909 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.720776 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 19:53:53.730825 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.730093 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6c72" Sep 10 19:53:53.740231 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.740210 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6c72" Sep 10 19:53:53.765602 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.765555 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerStarted","Data":"39fcfb9a6b9d5d4590e004e31ee5437be88731391c33dae04c9f8b2dda55fd82"} Sep 10 19:53:53.767549 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.767523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" event={"ID":"48adfbd41af55e45ba98e8777ebfc093","Type":"ContainerStarted","Data":"69e58a69f274ae04b1cf0f955510fc5e75ade5b619e02bc5a99b721a00ab475a"} Sep 10 19:53:53.794999 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.794925 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 19:53:53.808373 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808339 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c2a39fa-8215-4571-aa00-cdcd08fc3087-host\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.808517 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808386 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjtk\" (UniqueName: \"kubernetes.io/projected/1c2a39fa-8215-4571-aa00-cdcd08fc3087-kube-api-access-bbjtk\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.808517 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808439 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-systemd\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.808517 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808471 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-cni-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.808517 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-cni-binary-copy\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-hostroot\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808548 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cnibin\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-cni-netd\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808599 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovnkube-config\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808625 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-modprobe-d\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808655 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-kubelet\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-etc-kubernetes\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808709 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.808733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808732 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-env-overrides\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808759 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-kubernetes\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808772 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-hostroot\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc205713-3fae-42e9-a986-d9819f5e274b-tmp\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808851 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-etc-kubernetes\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.808886 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-kubelet\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809104 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-cni-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-socket-dir-parent\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-k8s-cni-cncf-io\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809260 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809239 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-socket-dir-parent\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809275 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-k8s-cni-cncf-io\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-conf-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809329 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-conf-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809376 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-ovn\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809406 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-lib-modules\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a17871d-91b4-42ed-90ae-2feee6eb985a-tmp-dir\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfx82\" (UniqueName: \"kubernetes.io/projected/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-kube-api-access-tfx82\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809483 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809518 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-systemd-units\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809559 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809585 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2450cbac-1850-4128-9a10-68d6d582c549-agent-certs\") pod \"konnectivity-agent-gxpgv\" (UID: \"2450cbac-1850-4128-9a10-68d6d582c549\") " pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809612 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-sys-fs\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809634 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-sys\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809667 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-node-log\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.809694 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809693 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809730 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-systemd\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz68h\" (UniqueName: \"kubernetes.io/projected/93c3aa0d-2854-4c0f-829d-d9125011599c-kube-api-access-wz68h\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809828 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-host\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809857 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-netns\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809898 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-multus-certs\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809952 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-multus-certs\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2cd9622-ed11-4e4c-b30a-7d94dc097090-dbus\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.809980 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-run-netns\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-etc-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810049 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2cd9622-ed11-4e4c-b30a-7d94dc097090-dbus\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810071 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-cni-bin\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810126 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-device-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810157 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-cni-multus\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810195 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810212 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-cni-multus\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.810476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76krq\" (UniqueName: \"kubernetes.io/projected/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-kube-api-access-76krq\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-registration-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810288 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-cni-binary-copy\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.810314 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810292 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a17871d-91b4-42ed-90ae-2feee6eb985a-hosts-file\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-cnibin\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.810386 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:53:54.310363911 +0000 UTC m=+3.031311032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810397 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-cnibin\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-host-slash\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810436 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-host-slash\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810441 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810487 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovn-node-metrics-cert\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810511 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810537 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysctl-conf\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810566 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-daemon-config\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810591 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2cd9622-ed11-4e4c-b30a-7d94dc097090-kubelet-config\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810656 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2cd9622-ed11-4e4c-b30a-7d94dc097090-kubelet-config\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:53.811325 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810693 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-run-netns\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810723 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2450cbac-1850-4128-9a10-68d6d582c549-konnectivity-ca\") pod \"konnectivity-agent-gxpgv\" (UID: \"2450cbac-1850-4128-9a10-68d6d582c549\") " pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-system-cni-dir\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-run\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810889 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-system-cni-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810914 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810940 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-kubelet\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-log-socket\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810972 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-multus-daemon-config\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.810990 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovnkube-script-lib\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysctl-d\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811024 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-system-cni-dir\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmlf\" (UniqueName: \"kubernetes.io/projected/fc205713-3fae-42e9-a986-d9819f5e274b-kube-api-access-zkmlf\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811117 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-cni-bin\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811155 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:53.812204 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811163 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-host-var-lib-cni-bin\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811186 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpgdd\" (UniqueName: \"kubernetes.io/projected/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-kube-api-access-vpgdd\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811250 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc205713-3fae-42e9-a986-d9819f5e274b-etc-tuned\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811274 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-slash\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811300 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-var-lib-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811324 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-socket-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811346 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysconfig\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-var-lib-kubelet\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811404 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c2a39fa-8215-4571-aa00-cdcd08fc3087-serviceca\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7wpw\" (UniqueName: \"kubernetes.io/projected/921cab07-9428-4d14-82ad-8d7cbbaa339a-kube-api-access-p7wpw\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811476 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp52x\" (UniqueName: \"kubernetes.io/projected/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-kube-api-access-cp52x\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpt69\" (UniqueName: \"kubernetes.io/projected/1a17871d-91b4-42ed-90ae-2feee6eb985a-kube-api-access-bpt69\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.811537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-iptables-alerter-script\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.813047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.812136 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-iptables-alerter-script\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.818472 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.818443 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 19:53:53.819310 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.819286 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:53:53.819420 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.819319 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:53:53.819420 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.819334 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:53.819420 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.819399 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:53:54.319380237 +0000 UTC m=+3.040327359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:53.823458 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.823432 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76krq\" (UniqueName: \"kubernetes.io/projected/7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b-kube-api-access-76krq\") pod \"iptables-alerter-k4qdv\" (UID: \"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b\") " pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:53.824003 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.823980 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfx82\" (UniqueName: \"kubernetes.io/projected/319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24-kube-api-access-tfx82\") pod \"multus-bqpkk\" (UID: \"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24\") " pod="openshift-multus/multus-bqpkk" Sep 10 19:53:53.911953 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.911916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.912166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.911965 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-etc-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.911993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-cni-bin\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912041 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-device-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.912166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912089 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-registration-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.912166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912113 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a17871d-91b4-42ed-90ae-2feee6eb985a-hosts-file\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.912166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912162 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovn-node-metrics-cert\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912248 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912269 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-etc-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysctl-conf\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912318 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-cni-bin\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-run-netns\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912372 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2450cbac-1850-4128-9a10-68d6d582c549-konnectivity-ca\") pod \"konnectivity-agent-gxpgv\" (UID: \"2450cbac-1850-4128-9a10-68d6d582c549\") " pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912378 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-run-netns\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912396 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-system-cni-dir\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912413 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.912454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-run\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912467 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-kubelet\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912501 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-log-socket\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912543 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovnkube-script-lib\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912550 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysctl-d\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmlf\" (UniqueName: \"kubernetes.io/projected/fc205713-3fae-42e9-a986-d9819f5e274b-kube-api-access-zkmlf\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912636 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-device-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912638 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysctl-conf\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpgdd\" (UniqueName: \"kubernetes.io/projected/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-kube-api-access-vpgdd\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc205713-3fae-42e9-a986-d9819f5e274b-etc-tuned\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912716 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a17871d-91b4-42ed-90ae-2feee6eb985a-hosts-file\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912722 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-system-cni-dir\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912732 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-slash\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912763 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-var-lib-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912772 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-registration-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-socket-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysconfig\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-var-lib-kubelet\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c2a39fa-8215-4571-aa00-cdcd08fc3087-serviceca\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912907 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysctl-d\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912922 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7wpw\" (UniqueName: \"kubernetes.io/projected/921cab07-9428-4d14-82ad-8d7cbbaa339a-kube-api-access-p7wpw\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912947 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-kubelet\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912948 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp52x\" (UniqueName: \"kubernetes.io/projected/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-kube-api-access-cp52x\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912990 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-log-socket\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.912990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpt69\" (UniqueName: \"kubernetes.io/projected/1a17871d-91b4-42ed-90ae-2feee6eb985a-kube-api-access-bpt69\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c2a39fa-8215-4571-aa00-cdcd08fc3087-host\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjtk\" (UniqueName: \"kubernetes.io/projected/1c2a39fa-8215-4571-aa00-cdcd08fc3087-kube-api-access-bbjtk\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913094 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-systemd\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913171 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cnibin\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-cni-netd\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913234 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovnkube-config\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.913868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-modprobe-d\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-env-overrides\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-kubernetes\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc205713-3fae-42e9-a986-d9819f5e274b-tmp\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913435 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cnibin\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-ovn\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-lib-modules\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.913534 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a17871d-91b4-42ed-90ae-2feee6eb985a-tmp-dir\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913653 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913680 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-systemd-units\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913730 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2450cbac-1850-4128-9a10-68d6d582c549-agent-certs\") pod \"konnectivity-agent-gxpgv\" (UID: \"2450cbac-1850-4128-9a10-68d6d582c549\") " pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913858 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-modprobe-d\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913897 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-sys-fs\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-sys\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.914780 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913957 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-node-log\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2450cbac-1850-4128-9a10-68d6d582c549-konnectivity-ca\") pod \"konnectivity-agent-gxpgv\" (UID: \"2450cbac-1850-4128-9a10-68d6d582c549\") " pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913966 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-run\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913983 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-systemd\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914026 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-sysconfig\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914040 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz68h\" (UniqueName: \"kubernetes.io/projected/93c3aa0d-2854-4c0f-829d-d9125011599c-kube-api-access-wz68h\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914065 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-slash\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914067 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-host\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914107 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-host\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914176 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-var-lib-openvswitch\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914217 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-cni-netd\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914216 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-node-log\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c2a39fa-8215-4571-aa00-cdcd08fc3087-host\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914302 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-sys-fs\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914334 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93c3aa0d-2854-4c0f-829d-d9125011599c-socket-dir\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.914344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-sys\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.915705 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:53.914373 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:53:54.41435468 +0000 UTC m=+3.135301788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.915801 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-env-overrides\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.916861 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovnkube-script-lib\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.916918 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-ovn\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.917245 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovnkube-config\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.917755 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.917772 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-kubernetes\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.917836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-etc-systemd\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.918598 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c2a39fa-8215-4571-aa00-cdcd08fc3087-serviceca\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.918682 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-var-lib-kubelet\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.918806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc205713-3fae-42e9-a986-d9819f5e274b-lib-modules\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.919075 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a17871d-91b4-42ed-90ae-2feee6eb985a-tmp-dir\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.919534 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-systemd-units\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.913679 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-run-systemd\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.920015 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.920099 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.920777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.920363 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.921530 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.921218 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc205713-3fae-42e9-a986-d9819f5e274b-tmp\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.922587 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.922565 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2450cbac-1850-4128-9a10-68d6d582c549-agent-certs\") pod \"konnectivity-agent-gxpgv\" (UID: \"2450cbac-1850-4128-9a10-68d6d582c549\") " pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:53.924819 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.924496 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjtk\" (UniqueName: \"kubernetes.io/projected/1c2a39fa-8215-4571-aa00-cdcd08fc3087-kube-api-access-bbjtk\") pod \"node-ca-dspv5\" (UID: \"1c2a39fa-8215-4571-aa00-cdcd08fc3087\") " pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:53.924819 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.924603 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmlf\" (UniqueName: \"kubernetes.io/projected/fc205713-3fae-42e9-a986-d9819f5e274b-kube-api-access-zkmlf\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.926287 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.926138 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.926287 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.926215 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc205713-3fae-42e9-a986-d9819f5e274b-etc-tuned\") pod \"tuned-4crnq\" (UID: \"fc205713-3fae-42e9-a986-d9819f5e274b\") " pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:53.926998 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.926971 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-ovn-node-metrics-cert\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:53.928975 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.928947 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz68h\" (UniqueName: \"kubernetes.io/projected/93c3aa0d-2854-4c0f-829d-d9125011599c-kube-api-access-wz68h\") pod \"aws-ebs-csi-driver-node-tsqlz\" (UID: \"93c3aa0d-2854-4c0f-829d-d9125011599c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:53.929077 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.928950 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpgdd\" (UniqueName: \"kubernetes.io/projected/7c2cfebb-5996-49e3-9ef1-0fd3583b649d-kube-api-access-vpgdd\") pod \"multus-additional-cni-plugins-jdtfj\" (UID: \"7c2cfebb-5996-49e3-9ef1-0fd3583b649d\") " pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:53.929758 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.929735 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7wpw\" (UniqueName: \"kubernetes.io/projected/921cab07-9428-4d14-82ad-8d7cbbaa339a-kube-api-access-p7wpw\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:53.930319 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.930297 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpt69\" (UniqueName: \"kubernetes.io/projected/1a17871d-91b4-42ed-90ae-2feee6eb985a-kube-api-access-bpt69\") pod \"node-resolver-tpt2k\" (UID: \"1a17871d-91b4-42ed-90ae-2feee6eb985a\") " pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:53.930643 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:53.930570 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp52x\" (UniqueName: \"kubernetes.io/projected/d1ec92e5-fc94-4afe-b064-8d417fd26a6e-kube-api-access-cp52x\") pod \"ovnkube-node-tszp7\" (UID: \"d1ec92e5-fc94-4afe-b064-8d417fd26a6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:54.007900 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.007865 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4qdv" Sep 10 19:53:54.015510 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.015486 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bqpkk" Sep 10 19:53:54.021458 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:54.021427 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9c4ea1_f2db_4550_81a3_f3e96e7cd45b.slice/crio-7cf1931479176afa55535f49a1eab4628e6ccd44164c8d0153a46f9ef21c8b2b WatchSource:0}: Error finding container 7cf1931479176afa55535f49a1eab4628e6ccd44164c8d0153a46f9ef21c8b2b: Status 404 returned error can't find the container with id 7cf1931479176afa55535f49a1eab4628e6ccd44164c8d0153a46f9ef21c8b2b Sep 10 19:53:54.026333 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:54.026305 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319df8bf_7a76_4ed1_a2a7_e9b78bf6ce24.slice/crio-97aa46d20de1cdb7d0a6423294ddfd0ce09c85adda54203d2d2f374e3492d37b WatchSource:0}: Error finding container 97aa46d20de1cdb7d0a6423294ddfd0ce09c85adda54203d2d2f374e3492d37b: Status 404 returned error can't find the container with id 97aa46d20de1cdb7d0a6423294ddfd0ce09c85adda54203d2d2f374e3492d37b Sep 10 19:53:54.027913 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.027894 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:53:54.035032 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.035005 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:53:54.043146 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.043118 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4crnq" Sep 10 19:53:54.058264 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.058236 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tpt2k" Sep 10 19:53:54.068264 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.068241 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" Sep 10 19:53:54.076142 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.076119 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dspv5" Sep 10 19:53:54.084160 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.084136 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" Sep 10 19:53:54.316161 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.316121 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:54.316374 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.316321 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:54.316449 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.316411 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:53:55.316387751 +0000 UTC m=+4.037334871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:54.416883 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.416840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:54.417042 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.416911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:54.417042 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.417003 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:54.417126 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.417066 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:53:54.417126 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.417078 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:53:55.417062222 +0000 UTC m=+4.138009326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:54.417126 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.417087 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:53:54.417126 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.417114 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:54.417267 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:54.417145 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:53:55.417133478 +0000 UTC m=+4.138080591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:54.744690 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.744649 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:48:53 +0000 UTC" deadline="2028-02-28 23:01:44.347567951 +0000 UTC" Sep 10 19:53:54.744690 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.744691 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12867h7m49.602882605s" Sep 10 19:53:54.771731 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.771689 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bqpkk" event={"ID":"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24","Type":"ContainerStarted","Data":"97aa46d20de1cdb7d0a6423294ddfd0ce09c85adda54203d2d2f374e3492d37b"} Sep 10 19:53:54.773215 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:54.773189 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4qdv" event={"ID":"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b","Type":"ContainerStarted","Data":"7cf1931479176afa55535f49a1eab4628e6ccd44164c8d0153a46f9ef21c8b2b"} Sep 10 19:53:55.241354 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:55.241280 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c3aa0d_2854_4c0f_829d_d9125011599c.slice/crio-631961300c76e74bbe50cca4ffb8a5d60878eb125e98f4a08f37c823fb160bbd WatchSource:0}: Error finding container 631961300c76e74bbe50cca4ffb8a5d60878eb125e98f4a08f37c823fb160bbd: Status 404 returned error can't find the container with id 631961300c76e74bbe50cca4ffb8a5d60878eb125e98f4a08f37c823fb160bbd Sep 10 19:53:55.252764 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:55.250953 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a17871d_91b4_42ed_90ae_2feee6eb985a.slice/crio-ad68dc8cab45f2b194ecac7076ce16a9ae82804eb03168aeaffe462234661df7 WatchSource:0}: Error finding container ad68dc8cab45f2b194ecac7076ce16a9ae82804eb03168aeaffe462234661df7: Status 404 returned error can't find the container with id ad68dc8cab45f2b194ecac7076ce16a9ae82804eb03168aeaffe462234661df7 Sep 10 19:53:55.261589 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:53:55.261100 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2cfebb_5996_49e3_9ef1_0fd3583b649d.slice/crio-81def67e8fe94915c03338f92fdf3651a5c80eb752184a6c61c3f17421966f31 WatchSource:0}: Error finding container 81def67e8fe94915c03338f92fdf3651a5c80eb752184a6c61c3f17421966f31: Status 404 returned error can't find the container with id 81def67e8fe94915c03338f92fdf3651a5c80eb752184a6c61c3f17421966f31 Sep 10 19:53:55.324658 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.324635 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:55.324830 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.324809 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:55.324906 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.324892 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:53:57.324869612 +0000 UTC m=+6.045816725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:55.426274 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.426233 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:55.426426 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.426329 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:55.426501 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.426419 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:53:55.426501 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.426451 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:53:55.426501 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.426467 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:55.426501 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.426473 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:55.426698 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.426547 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:53:57.426523389 +0000 UTC m=+6.147470505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:55.426698 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.426569 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:53:57.426558599 +0000 UTC m=+6.147505698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:55.745263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.745014 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:48:53 +0000 UTC" deadline="2028-03-13 23:32:34.301586109 +0000 UTC" Sep 10 19:53:55.745263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.745258 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13203h38m38.556334202s" Sep 10 19:53:55.759539 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.759513 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:55.759725 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.759513 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:55.759817 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.759513 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:55.759862 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.759823 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:53:55.759900 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.759680 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:53:55.759936 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:55.759911 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:53:55.776509 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.776445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4crnq" event={"ID":"fc205713-3fae-42e9-a986-d9819f5e274b","Type":"ContainerStarted","Data":"9e14e9aab829a20d91a8e5eac5f5cf9e21a86c12af7341951569537d07498d70"} Sep 10 19:53:55.778057 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.778025 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dspv5" event={"ID":"1c2a39fa-8215-4571-aa00-cdcd08fc3087","Type":"ContainerStarted","Data":"3638333ce14b0e7fed0ecee068d36b6150c03cdb37eb0b9d093faea7e20a6818"} Sep 10 19:53:55.779450 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.779398 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" event={"ID":"93c3aa0d-2854-4c0f-829d-d9125011599c","Type":"ContainerStarted","Data":"631961300c76e74bbe50cca4ffb8a5d60878eb125e98f4a08f37c823fb160bbd"} Sep 10 19:53:55.782053 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.782027 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" event={"ID":"48adfbd41af55e45ba98e8777ebfc093","Type":"ContainerStarted","Data":"389dafc552421658c33b72c56f80804daade5a7600c6234fe76905bfe825ca9c"} Sep 10 19:53:55.785498 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.785377 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"8fe0c5519c8fc707dd15b99078f20e1aea77dbbe3e5bda76f095838e25d7128f"} Sep 10 19:53:55.787952 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.787921 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tpt2k" event={"ID":"1a17871d-91b4-42ed-90ae-2feee6eb985a","Type":"ContainerStarted","Data":"ad68dc8cab45f2b194ecac7076ce16a9ae82804eb03168aeaffe462234661df7"} Sep 10 19:53:55.789996 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.789972 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerStarted","Data":"ae72b07bfeeeced9afe29198b87dd0f67ec4cf149c7c693199ac056374b1a01c"} Sep 10 19:53:55.791531 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.791508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerStarted","Data":"81def67e8fe94915c03338f92fdf3651a5c80eb752184a6c61c3f17421966f31"} Sep 10 19:53:55.792972 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.792947 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gxpgv" event={"ID":"2450cbac-1850-4128-9a10-68d6d582c549","Type":"ContainerStarted","Data":"4629db3e4af8d9e9c9b924d6ab0a1d2b91b2c2c068c789b5e14b4a481fd8eca1"} Sep 10 19:53:55.797803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:55.797759 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" podStartSLOduration=2.797748601 podStartE2EDuration="2.797748601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:55.797677633 +0000 UTC m=+4.518624755" watchObservedRunningTime="2026-09-10 19:53:55.797748601 +0000 UTC m=+4.518695849" Sep 10 19:53:56.800128 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:56.800090 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tpt2k" event={"ID":"1a17871d-91b4-42ed-90ae-2feee6eb985a","Type":"ContainerStarted","Data":"a7578a22908522ef5a51a1f2000b4ffa34e7b5e25003cd52dc05f7e12974fe80"} Sep 10 19:53:56.805347 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:56.805315 2570 generic.go:356] "Generic (PLEG): container finished" podID="667ccdd66cb316e2e58a7c2fdcf49d11" containerID="ae72b07bfeeeced9afe29198b87dd0f67ec4cf149c7c693199ac056374b1a01c" exitCode=0 Sep 10 19:53:56.805547 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:56.805407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerDied","Data":"ae72b07bfeeeced9afe29198b87dd0f67ec4cf149c7c693199ac056374b1a01c"} Sep 10 19:53:56.816977 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:56.816930 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tpt2k" podStartSLOduration=3.840927469 podStartE2EDuration="4.816911152s" totalImagesPullingTime="975.983683ms" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:56.816327398 +0000 UTC m=+5.537274522" watchObservedRunningTime="2026-09-10 19:53:56.816911152 +0000 UTC m=+5.537858322" Sep 10 19:53:57.345538 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.344968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:57.345538 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.345144 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:57.345538 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.345206 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:01.345188795 +0000 UTC m=+10.066135894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:53:57.445914 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.445833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:57.446112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.445999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:57.446185 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.446163 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:53:57.446185 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.446182 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:53:57.446282 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.446196 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:57.446282 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.446272 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:01.446254562 +0000 UTC m=+10.167201672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:53:57.446755 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.446737 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:57.446820 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.446793 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:01.44677738 +0000 UTC m=+10.167724480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:53:57.760216 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.760181 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:57.760391 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.760307 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:53:57.760714 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.760691 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:57.760826 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.760804 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:53:57.762959 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.762936 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:57.763064 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:57.763050 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:53:57.816308 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:57.816273 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"667ccdd66cb316e2e58a7c2fdcf49d11","Type":"ContainerStarted","Data":"f1b70809cadfb75653f062194b26d16d0d860f1f33bc139a071e74247c9c312a"} Sep 10 19:53:58.822305 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:58.822256 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4qdv" event={"ID":"7f9c4ea1-f2db-4550-81a3-f3e96e7cd45b","Type":"ContainerStarted","Data":"00d77712010c64fc81efe69d26f94591c218f5a680864b38fa386e5299da5c36"} Sep 10 19:53:58.840362 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:58.839202 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" podStartSLOduration=5.83915495 podStartE2EDuration="5.83915495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:57.832286887 +0000 UTC m=+6.553234017" watchObservedRunningTime="2026-09-10 19:53:58.83915495 +0000 UTC m=+7.560102071" Sep 10 19:53:59.760600 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:59.760560 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:53:59.760827 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:59.760560 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:53:59.760827 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:59.760688 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:53:59.760827 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:59.760798 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:53:59.760987 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:53:59.760862 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:53:59.760987 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:53:59.760935 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:01.381476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:01.381435 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:01.381951 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.381617 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:54:01.381951 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.381717 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:09.381694439 +0000 UTC m=+18.102641537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:54:01.481966 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:01.481852 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:01.481966 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:01.481914 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:01.482233 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.482033 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:54:01.482233 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.482050 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:54:01.482233 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.482089 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:54:01.482233 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.482103 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:54:01.482233 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.482132 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:09.482110648 +0000 UTC m=+18.203057759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:54:01.482233 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.482155 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:09.482143677 +0000 UTC m=+18.203090778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:54:01.759974 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:01.759942 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:01.760202 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.760050 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:01.760202 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:01.760099 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:01.760202 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:01.760122 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:01.760376 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.760220 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:01.760376 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:01.760328 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:03.761261 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:03.761208 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:03.761713 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:03.761454 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:03.762705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:03.762041 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:03.762705 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:03.762260 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:03.762705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:03.762462 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:03.762705 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:03.762698 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:05.759653 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:05.759617 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:05.760105 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:05.759626 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:05.760105 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:05.759751 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:05.760105 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:05.759865 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:05.760105 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:05.759892 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:05.760105 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:05.759949 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:07.759965 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:07.759923 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:07.759965 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:07.759960 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:07.760446 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:07.759923 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:07.760446 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:07.760057 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:07.760446 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:07.760164 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:07.760446 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:07.760227 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:09.444288 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:09.444039 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:09.444811 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.444197 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:54:09.444811 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.444369 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:25.444346072 +0000 UTC m=+34.165293171 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:54:09.545133 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:09.545103 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:09.545318 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:09.545181 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:09.545318 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.545298 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:54:09.545318 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.545304 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:54:09.545456 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.545328 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:54:09.545456 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.545343 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:54:09.545456 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.545363 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:25.545346516 +0000 UTC m=+34.266293614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:54:09.545456 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.545396 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:25.545380008 +0000 UTC m=+34.266327134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:54:09.760082 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:09.760029 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:09.760248 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:09.760029 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:09.760248 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.760184 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:09.760248 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.760232 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:09.760248 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:09.760031 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:09.760460 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:09.760331 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:11.760318 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:11.760286 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:11.760747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:11.760403 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:11.760747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:11.760522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:11.760747 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:11.760407 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:11.760747 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:11.760576 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:11.760747 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:11.760603 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:12.852653 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.852448 2570 generic.go:356] "Generic (PLEG): container finished" podID="7c2cfebb-5996-49e3-9ef1-0fd3583b649d" containerID="37e9435aab020854bd992362e46c4fe342f70b0f0a5a5e8602c3aae543083950" exitCode=0 Sep 10 19:54:12.853597 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.852537 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerDied","Data":"37e9435aab020854bd992362e46c4fe342f70b0f0a5a5e8602c3aae543083950"} Sep 10 19:54:12.854520 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.854422 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gxpgv" event={"ID":"2450cbac-1850-4128-9a10-68d6d582c549","Type":"ContainerStarted","Data":"74c3f72a1b73c0c3bf3e162b032c1ed5ca5187477ca3bfa0556d98bfc997ace2"} Sep 10 19:54:12.856128 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.856103 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4crnq" event={"ID":"fc205713-3fae-42e9-a986-d9819f5e274b","Type":"ContainerStarted","Data":"03c96ed1a0b4bb34c46c9bfcccc418f1c583da9545171bcf2f0320a5163a3a97"} Sep 10 19:54:12.857733 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.857709 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dspv5" event={"ID":"1c2a39fa-8215-4571-aa00-cdcd08fc3087","Type":"ContainerStarted","Data":"760bce0a7044245c19f153c9dda4f1490f3d03c16b3e451dfc643c2d58cd96d3"} Sep 10 19:54:12.859350 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.859329 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" event={"ID":"93c3aa0d-2854-4c0f-829d-d9125011599c","Type":"ContainerStarted","Data":"1edf19ac3b8b3092bb1b6ee8b0719e405665166ce2415af9bc6e60872117cae8"} Sep 10 19:54:12.861052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.861023 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bqpkk" event={"ID":"319df8bf-7a76-4ed1-a2a7-e9b78bf6ce24","Type":"ContainerStarted","Data":"7ae2181f4742d1adcffe815b5b2f122dc0e76eb405e6264fc9b8663d34d864e8"} Sep 10 19:54:12.864214 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.864194 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"75001c63a55429b6f26a2741b86be854b8c98df077ff31e326b31d6912c2287f"} Sep 10 19:54:12.864299 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.864268 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"34ca0d1161636408ccc3f5dded7e13f26fb3b5bf43a95632fd048bdf1f8d0de7"} Sep 10 19:54:12.864358 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.864328 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"045bacc7229d74c90eae80ce0f737247eb9fbe961e988b8001b258614496e891"} Sep 10 19:54:12.864393 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.864376 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"82a3b95968195ccce9f4dab67f4dd08f4a025edda1ae4796fe1a7902050ca3de"} Sep 10 19:54:12.864432 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.864423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"d1156c788e6b45372125e46a0e7464d2fc52123486bc9ee5c6153236424701ad"} Sep 10 19:54:12.864482 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.864470 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"8194308d721b0d5b92f113c3d05f3bb21f7ef4d7256710b16e7f9bfadced7bf2"} Sep 10 19:54:12.874636 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.874594 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k4qdv" podStartSLOduration=18.668871228 podStartE2EDuration="20.87458204s" totalImagesPullingTime="2.205710812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:58.840344932 +0000 UTC m=+7.561292053" watchObservedRunningTime="2026-09-10 19:54:12.87458204 +0000 UTC m=+21.595529208" Sep 10 19:54:12.891175 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.889754 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gxpgv" podStartSLOduration=4.459100822 podStartE2EDuration="20.889736523s" totalImagesPullingTime="16.430635701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:12.888287642 +0000 UTC m=+21.609234765" watchObservedRunningTime="2026-09-10 19:54:12.889736523 +0000 UTC m=+21.610683642" Sep 10 19:54:12.906896 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.906854 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4crnq" podStartSLOduration=4.226873954 podStartE2EDuration="20.906840566s" totalImagesPullingTime="16.679966612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:12.906790979 +0000 UTC m=+21.627738100" watchObservedRunningTime="2026-09-10 19:54:12.906840566 +0000 UTC m=+21.627787686" Sep 10 19:54:12.920837 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:12.920803 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dspv5" podStartSLOduration=4.253459808 podStartE2EDuration="20.920790572s" totalImagesPullingTime="16.667330764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:12.920418678 +0000 UTC m=+21.641365798" watchObservedRunningTime="2026-09-10 19:54:12.920790572 +0000 UTC m=+21.641737688" Sep 10 19:54:13.078971 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.078828 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 19:54:13.758188 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.758087 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T19:54:13.078966446Z","UUID":"90fbc554-b678-4823-a305-06d720ff5f80","Handler":null,"Name":"","Endpoint":""} Sep 10 19:54:13.759254 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.759228 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:13.759392 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:13.759335 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:13.759392 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.759327 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:13.759392 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.759377 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:13.759533 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:13.759465 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:13.759584 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:13.759566 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:13.759784 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.759763 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 19:54:13.759880 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.759796 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 19:54:13.868333 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:13.868302 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" event={"ID":"93c3aa0d-2854-4c0f-829d-d9125011599c","Type":"ContainerStarted","Data":"d6ca04d7d93802fcf4921adaddc8e2d554d66b7b4bebc6a92f38a004c1c2f704"} Sep 10 19:54:14.874778 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:14.874740 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"c7bf3121e5fc4fbb28964ccd55282d94bfb17cc2319cf34db281d5e772cf7d36"} Sep 10 19:54:14.878020 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:14.877980 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" event={"ID":"93c3aa0d-2854-4c0f-829d-d9125011599c","Type":"ContainerStarted","Data":"72f8f6920a7577c4adbe395af91037bed50bbcb5818f252778dd712d2dfbfb43"} Sep 10 19:54:14.895695 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:14.895654 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bqpkk" podStartSLOduration=4.969598748 podStartE2EDuration="22.895641651s" totalImagesPullingTime="17.926042903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:12.948685195 +0000 UTC m=+21.669632313" watchObservedRunningTime="2026-09-10 19:54:14.895641651 +0000 UTC m=+23.616588769" Sep 10 19:54:14.895957 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:14.895939 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsqlz" podStartSLOduration=4.509092492 podStartE2EDuration="22.895935067s" totalImagesPullingTime="18.386842575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:14.895520556 +0000 UTC m=+23.616467689" watchObservedRunningTime="2026-09-10 19:54:14.895935067 +0000 UTC m=+23.616882214" Sep 10 19:54:15.762574 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:15.762540 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:15.762841 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:15.762540 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:15.762841 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:15.762649 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:15.762841 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:15.762735 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:15.762841 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:15.762542 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:15.762841 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:15.762829 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:16.449416 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:16.449382 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:54:16.450043 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:16.450021 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:54:17.759392 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.759358 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:17.759788 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.759373 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:17.759788 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.759477 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:17.759788 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:17.759454 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:17.759788 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:17.759556 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:17.759788 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:17.759653 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:17.890474 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.890374 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" event={"ID":"d1ec92e5-fc94-4afe-b064-8d417fd26a6e","Type":"ContainerStarted","Data":"1dc35d0a4ca75093a8d863c176673535f24885395f93ad47ffcd28a1f9a00b74"} Sep 10 19:54:17.891182 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.890897 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:54:17.891182 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.891099 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:54:17.912932 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.912798 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:54:17.913742 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.913492 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:54:17.923324 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:17.923178 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" podStartSLOduration=9.067705376 podStartE2EDuration="25.923160609s" totalImagesPullingTime="16.855455233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:17.922468389 +0000 UTC m=+26.643415511" watchObservedRunningTime="2026-09-10 19:54:17.923160609 +0000 UTC m=+26.644107729" Sep 10 19:54:18.894816 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:18.894629 2570 generic.go:356] "Generic (PLEG): container finished" podID="7c2cfebb-5996-49e3-9ef1-0fd3583b649d" containerID="66726423e9ac61717c1f57ea9f464e27a3641cc074dfb0d9fd70521bf4d00707" exitCode=0 Sep 10 19:54:18.895441 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:18.894708 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerDied","Data":"66726423e9ac61717c1f57ea9f464e27a3641cc074dfb0d9fd70521bf4d00707"} Sep 10 19:54:18.895441 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:18.894916 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 19:54:19.750663 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.750625 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2lgfz"] Sep 10 19:54:19.750977 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.750763 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:19.750977 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:19.750843 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:19.756363 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.756281 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flfcj"] Sep 10 19:54:19.756491 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.756389 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:19.756543 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:19.756485 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:19.759997 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.759965 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:19.760135 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:19.760105 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:19.763454 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.763432 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dhzfs"] Sep 10 19:54:19.896490 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.896454 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:19.896867 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:19.896552 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 19:54:19.896867 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:19.896594 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:20.903664 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:20.901165 2570 generic.go:356] "Generic (PLEG): container finished" podID="7c2cfebb-5996-49e3-9ef1-0fd3583b649d" containerID="e189e34958119f9f3f2fa2a70968a509060d6fd6850e91bb091edd019d0674fc" exitCode=0 Sep 10 19:54:20.903664 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:20.901222 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerDied","Data":"e189e34958119f9f3f2fa2a70968a509060d6fd6850e91bb091edd019d0674fc"} Sep 10 19:54:21.760214 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:21.760186 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:21.760368 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:21.760267 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:21.760368 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:21.760285 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:21.760368 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:21.760339 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:21.760525 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:21.760406 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:21.760525 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:21.760506 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:21.906230 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:21.906026 2570 generic.go:356] "Generic (PLEG): container finished" podID="7c2cfebb-5996-49e3-9ef1-0fd3583b649d" containerID="712f0efc21474dcfdead73f574f54bed065c3ddfa79a35e9637e97662ca96984" exitCode=0 Sep 10 19:54:21.906653 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:21.906111 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerDied","Data":"712f0efc21474dcfdead73f574f54bed065c3ddfa79a35e9637e97662ca96984"} Sep 10 19:54:22.451990 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:22.451961 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:54:22.452233 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:22.452215 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 19:54:22.463207 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:22.463175 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" podUID="d1ec92e5-fc94-4afe-b064-8d417fd26a6e" containerName="ovnkube-controller" probeResult="failure" output="" Sep 10 19:54:22.472684 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:22.472654 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" podUID="d1ec92e5-fc94-4afe-b064-8d417fd26a6e" containerName="ovnkube-controller" probeResult="failure" output="" Sep 10 19:54:23.108583 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:23.108547 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:54:23.109010 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:23.108740 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 19:54:23.109275 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:23.109255 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gxpgv" Sep 10 19:54:23.759286 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:23.759256 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:23.759286 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:23.759274 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:23.759600 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:23.759373 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flfcj" podUID="c2cd9622-ed11-4e4c-b30a-7d94dc097090" Sep 10 19:54:23.759600 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:23.759435 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzfs" podUID="921cab07-9428-4d14-82ad-8d7cbbaa339a" Sep 10 19:54:23.759600 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:23.759462 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:23.759600 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:23.759548 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lgfz" podUID="99b74afe-5b35-437d-b27f-2ed325f8e7c8" Sep 10 19:54:24.570252 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.570223 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeReady" Sep 10 19:54:24.570783 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.570385 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 19:54:24.603592 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.603564 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6ff7548c8d-d2kxn"] Sep 10 19:54:24.626170 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.626144 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ff7548c8d-d2kxn"] Sep 10 19:54:24.626310 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.626186 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-glzkt"] Sep 10 19:54:24.626310 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.626240 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.628505 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.628483 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 19:54:24.628624 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.628557 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 19:54:24.628812 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.628798 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 19:54:24.628898 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.628867 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2xjwm\"" Sep 10 19:54:24.633151 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.633133 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 19:54:24.636827 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.636805 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tfpxv"] Sep 10 19:54:24.637649 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.637063 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.639468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.639373 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 19:54:24.639468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.639434 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bxvfx\"" Sep 10 19:54:24.639611 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.639429 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 19:54:24.640266 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.639579 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 19:54:24.646998 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.646963 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-glzkt"] Sep 10 19:54:24.647110 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.647007 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tfpxv"] Sep 10 19:54:24.647293 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.647276 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.649782 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.649759 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 19:54:24.649965 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.649765 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ngfwp\"" Sep 10 19:54:24.649965 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.649765 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 19:54:24.753409 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0a7eead-c04c-4105-82b5-d060fe76f2ef-cert\") pod \"ingress-canary-glzkt\" (UID: \"a0a7eead-c04c-4105-82b5-d060fe76f2ef\") " pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.753409 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753375 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq2fp\" (UniqueName: \"kubernetes.io/projected/a0a7eead-c04c-4105-82b5-d060fe76f2ef-kube-api-access-xq2fp\") pod \"ingress-canary-glzkt\" (UID: \"a0a7eead-c04c-4105-82b5-d060fe76f2ef\") " pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.753409 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753404 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753434 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e92499b-cb2e-4203-8247-f3ca5d908d4e-tmp-dir\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753481 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753506 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l858w\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-kube-api-access-l858w\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvx9d\" (UniqueName: \"kubernetes.io/projected/6e92499b-cb2e-4203-8247-f3ca5d908d4e-kube-api-access-cvx9d\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753565 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-installation-pull-secrets\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753591 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e92499b-cb2e-4203-8247-f3ca5d908d4e-config-volume\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.753644 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-image-registry-private-configuration\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753921 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753720 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-trusted-ca\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753921 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f26794b-b02c-4870-9cd2-0be56dfb642a-ca-trust-extracted\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753921 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753790 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-certificates\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.753921 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.753813 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-bound-sa-token\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.854901 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.854865 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq2fp\" (UniqueName: \"kubernetes.io/projected/a0a7eead-c04c-4105-82b5-d060fe76f2ef-kube-api-access-xq2fp\") pod \"ingress-canary-glzkt\" (UID: \"a0a7eead-c04c-4105-82b5-d060fe76f2ef\") " pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.854901 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.854904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.855112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.854923 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e92499b-cb2e-4203-8247-f3ca5d908d4e-tmp-dir\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.855112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855020 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l858w\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-kube-api-access-l858w\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855112 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:24.855032 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855116 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvx9d\" (UniqueName: \"kubernetes.io/projected/6e92499b-cb2e-4203-8247-f3ca5d908d4e-kube-api-access-cvx9d\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855144 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-installation-pull-secrets\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:24.855179 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:54:25.355154062 +0000 UTC m=+34.076101181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:24.855192 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:24.855211 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855224 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e92499b-cb2e-4203-8247-f3ca5d908d4e-config-volume\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855234 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e92499b-cb2e-4203-8247-f3ca5d908d4e-tmp-dir\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-image-registry-private-configuration\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:24.855266 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:25.355249918 +0000 UTC m=+34.076197033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-trusted-ca\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f26794b-b02c-4870-9cd2-0be56dfb642a-ca-trust-extracted\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855869 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-certificates\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855869 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855584 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-bound-sa-token\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.855869 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0a7eead-c04c-4105-82b5-d060fe76f2ef-cert\") pod \"ingress-canary-glzkt\" (UID: \"a0a7eead-c04c-4105-82b5-d060fe76f2ef\") " pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.855869 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855699 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e92499b-cb2e-4203-8247-f3ca5d908d4e-config-volume\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.855869 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.855750 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f26794b-b02c-4870-9cd2-0be56dfb642a-ca-trust-extracted\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.856103 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.856063 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-certificates\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.856382 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.856353 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-trusted-ca\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.859269 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.859246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-image-registry-private-configuration\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.859370 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.859273 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0a7eead-c04c-4105-82b5-d060fe76f2ef-cert\") pod \"ingress-canary-glzkt\" (UID: \"a0a7eead-c04c-4105-82b5-d060fe76f2ef\") " pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.859370 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.859294 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-installation-pull-secrets\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.863819 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.863796 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq2fp\" (UniqueName: \"kubernetes.io/projected/a0a7eead-c04c-4105-82b5-d060fe76f2ef-kube-api-access-xq2fp\") pod \"ingress-canary-glzkt\" (UID: \"a0a7eead-c04c-4105-82b5-d060fe76f2ef\") " pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:24.863897 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.863883 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-bound-sa-token\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.863958 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.863909 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvx9d\" (UniqueName: \"kubernetes.io/projected/6e92499b-cb2e-4203-8247-f3ca5d908d4e-kube-api-access-cvx9d\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:24.864420 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.864389 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l858w\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-kube-api-access-l858w\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:24.950541 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:24.950499 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-glzkt" Sep 10 19:54:25.259773 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.259582 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-glzkt"] Sep 10 19:54:25.359927 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.359816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:25.359927 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.359880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:25.360238 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.359981 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:25.360238 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.359999 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:25.360238 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.360014 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:25.360238 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.360061 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:54:26.36003969 +0000 UTC m=+35.080986810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:25.360238 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.360101 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:26.360090877 +0000 UTC m=+35.081037978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:25.461011 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.460973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:25.461327 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.461158 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 19:54:25.461327 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.461221 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret podName:c2cd9622-ed11-4e4c-b30a-7d94dc097090 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:57.461203224 +0000 UTC m=+66.182150337 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret") pod "global-pull-secret-syncer-flfcj" (UID: "c2cd9622-ed11-4e4c-b30a-7d94dc097090") : object "kube-system"/"original-pull-secret" not registered Sep 10 19:54:25.562227 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.562175 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:25.562417 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.562329 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:25.562417 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.562357 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 19:54:25.562417 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.562382 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 19:54:25.562417 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.562395 2570 projected.go:196] Error preparing data for projected volume kube-api-access-vsvjq for pod openshift-network-diagnostics/network-check-target-2lgfz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:54:25.562627 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.562451 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:54:25.562627 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.562462 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq podName:99b74afe-5b35-437d-b27f-2ed325f8e7c8 nodeName:}" failed. No retries permitted until 2026-09-10 19:54:57.562443702 +0000 UTC m=+66.283390800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsvjq" (UniqueName: "kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq") pod "network-check-target-2lgfz" (UID: "99b74afe-5b35-437d-b27f-2ed325f8e7c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 19:54:25.562627 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:25.562499 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:57.562487564 +0000 UTC m=+66.283434660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 19:54:25.759584 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.759546 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:25.760415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.759546 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:25.760415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.759702 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:25.762169 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.762147 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 19:54:25.762306 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.762173 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mcqxm\"" Sep 10 19:54:25.762306 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.762198 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 19:54:25.762435 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.762420 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 19:54:25.762496 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.762469 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mr4n2\"" Sep 10 19:54:25.762982 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:25.762958 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 19:54:26.368908 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:26.368872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:26.369095 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:26.368927 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:26.369095 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:26.369034 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:26.369095 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:26.369037 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:26.369258 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:26.369109 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:26.369258 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:26.369135 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:54:28.369112142 +0000 UTC m=+37.090059253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:26.369258 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:26.369159 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:28.369146387 +0000 UTC m=+37.090093483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:26.748930 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:54:26.748903 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a7eead_c04c_4105_82b5_d060fe76f2ef.slice/crio-a095c313bf926033ef8bc9979ff961f99e5dd9460d4a957fcbd3e76a0def1803 WatchSource:0}: Error finding container a095c313bf926033ef8bc9979ff961f99e5dd9460d4a957fcbd3e76a0def1803: Status 404 returned error can't find the container with id a095c313bf926033ef8bc9979ff961f99e5dd9460d4a957fcbd3e76a0def1803 Sep 10 19:54:26.918750 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:26.918717 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-glzkt" event={"ID":"a0a7eead-c04c-4105-82b5-d060fe76f2ef","Type":"ContainerStarted","Data":"a095c313bf926033ef8bc9979ff961f99e5dd9460d4a957fcbd3e76a0def1803"} Sep 10 19:54:27.924710 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:27.924672 2570 generic.go:356] "Generic (PLEG): container finished" podID="7c2cfebb-5996-49e3-9ef1-0fd3583b649d" containerID="afe4e50b5e5fd07437d2453c78dfeaaf726e55e579c02b7654489778dd286bd2" exitCode=0 Sep 10 19:54:27.925248 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:27.924731 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerDied","Data":"afe4e50b5e5fd07437d2453c78dfeaaf726e55e579c02b7654489778dd286bd2"} Sep 10 19:54:28.383865 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:28.383827 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:28.384031 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:28.383915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:28.384031 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:28.383984 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:28.384031 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:28.384004 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:28.384031 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:28.384004 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:28.384175 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:28.384062 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:54:32.384045156 +0000 UTC m=+41.104992275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:28.384175 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:28.384095 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:32.384086688 +0000 UTC m=+41.105033799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:28.930858 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:28.930820 2570 generic.go:356] "Generic (PLEG): container finished" podID="7c2cfebb-5996-49e3-9ef1-0fd3583b649d" containerID="adf98586043793750a8a9281775bf55c65ef0b43cc415752cd9dcacab5e184a0" exitCode=0 Sep 10 19:54:28.931270 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:28.930920 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerDied","Data":"adf98586043793750a8a9281775bf55c65ef0b43cc415752cd9dcacab5e184a0"} Sep 10 19:54:29.935205 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:29.934971 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-glzkt" event={"ID":"a0a7eead-c04c-4105-82b5-d060fe76f2ef","Type":"ContainerStarted","Data":"d4c07907babfe8db4fb1d7141fa69965e5f95ec912b70681d9664ede44b0eb5e"} Sep 10 19:54:29.938576 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:29.938547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" event={"ID":"7c2cfebb-5996-49e3-9ef1-0fd3583b649d","Type":"ContainerStarted","Data":"e5ebb503dc4a80aa6d2c58b01566e4548b34143e2a74738dd837c6430ad25e58"} Sep 10 19:54:29.954125 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:29.954055 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-glzkt" podStartSLOduration=2.989352437 podStartE2EDuration="5.954041035s" totalImagesPullingTime="2.964688598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:29.952955796 +0000 UTC m=+38.673902915" watchObservedRunningTime="2026-09-10 19:54:29.954041035 +0000 UTC m=+38.674988168" Sep 10 19:54:29.978146 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:29.978086 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jdtfj" podStartSLOduration=9.27287234 podStartE2EDuration="37.978056514s" totalImagesPullingTime="28.705184174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:54:29.977213804 +0000 UTC m=+38.698160923" watchObservedRunningTime="2026-09-10 19:54:29.978056514 +0000 UTC m=+38.699003632" Sep 10 19:54:30.140107 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:30.140009 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:31.527389 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:31.527352 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:32.416274 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:32.416241 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:32.416437 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:32.416292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:32.416437 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:32.416389 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:32.416437 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:32.416391 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:32.416534 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:32.416454 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:54:40.416439453 +0000 UTC m=+49.137386574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:32.416534 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:32.416400 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:32.416534 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:32.416518 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:40.416507121 +0000 UTC m=+49.137454235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:37.286799 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:37.286763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:37.386830 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:37.386795 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:37.488787 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:37.488750 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:38.049915 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:38.049866 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:38.134918 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:38.134873 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:38.213774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:38.213744 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:39.223548 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:39.223511 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:39.325238 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:39.325206 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" limit=1024 Sep 10 19:54:40.474322 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:40.474280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:40.474709 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:40.474333 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:40.474709 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:40.474416 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:40.474709 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:40.474445 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:40.474709 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:40.474456 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:40.474709 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:40.474487 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:54:56.474471052 +0000 UTC m=+65.195418149 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:40.474709 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:40.474502 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:54:56.474495998 +0000 UTC m=+65.195443095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:52.472626 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:52.472599 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tszp7" Sep 10 19:54:56.490461 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:56.490411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:54:56.490461 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:56.490474 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:54:56.491003 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:56.490558 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 19:54:56.491003 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:56.490570 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6ff7548c8d-d2kxn: secret "image-registry-tls" not found Sep 10 19:54:56.491003 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:56.490622 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls podName:8f26794b-b02c-4870-9cd2-0be56dfb642a nodeName:}" failed. No retries permitted until 2026-09-10 19:55:28.490608913 +0000 UTC m=+97.211556038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls") pod "image-registry-6ff7548c8d-d2kxn" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a") : secret "image-registry-tls" not found Sep 10 19:54:56.491003 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:56.490557 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 19:54:56.491003 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:56.490697 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls podName:6e92499b-cb2e-4203-8247-f3ca5d908d4e nodeName:}" failed. No retries permitted until 2026-09-10 19:55:28.490683362 +0000 UTC m=+97.211630460 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls") pod "dns-default-tfpxv" (UID: "6e92499b-cb2e-4203-8247-f3ca5d908d4e") : secret "dns-default-metrics-tls" not found Sep 10 19:54:57.497315 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.497267 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:57.499825 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.499805 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 19:54:57.511485 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.511457 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cd9622-ed11-4e4c-b30a-7d94dc097090-original-pull-secret\") pod \"global-pull-secret-syncer-flfcj\" (UID: \"c2cd9622-ed11-4e4c-b30a-7d94dc097090\") " pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:57.578712 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.578671 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flfcj" Sep 10 19:54:57.598598 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.598570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:54:57.598707 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.598625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:57.601177 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.601154 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 19:54:57.601286 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.601234 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 19:54:57.609044 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:57.609024 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 19:54:57.609144 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:54:57.609108 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs podName:921cab07-9428-4d14-82ad-8d7cbbaa339a nodeName:}" failed. No retries permitted until 2026-09-10 19:56:01.609090945 +0000 UTC m=+130.330038056 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs") pod "network-metrics-daemon-dhzfs" (UID: "921cab07-9428-4d14-82ad-8d7cbbaa339a") : secret "metrics-daemon-secret" not found Sep 10 19:54:57.611145 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.611128 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 19:54:57.622326 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.622302 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsvjq\" (UniqueName: \"kubernetes.io/projected/99b74afe-5b35-437d-b27f-2ed325f8e7c8-kube-api-access-vsvjq\") pod \"network-check-target-2lgfz\" (UID: \"99b74afe-5b35-437d-b27f-2ed325f8e7c8\") " pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:57.844643 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.844569 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flfcj"] Sep 10 19:54:57.847942 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:54:57.847908 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2cd9622_ed11_4e4c_b30a_7d94dc097090.slice/crio-d1cfdece3c511f9058817bb342b8e5c2daa92b3feeae3c1adaf55fe40eb32e9f WatchSource:0}: Error finding container d1cfdece3c511f9058817bb342b8e5c2daa92b3feeae3c1adaf55fe40eb32e9f: Status 404 returned error can't find the container with id d1cfdece3c511f9058817bb342b8e5c2daa92b3feeae3c1adaf55fe40eb32e9f Sep 10 19:54:57.874876 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.874857 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mcqxm\"" Sep 10 19:54:57.882856 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:57.882825 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:54:58.005112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:58.005059 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flfcj" event={"ID":"c2cd9622-ed11-4e4c-b30a-7d94dc097090","Type":"ContainerStarted","Data":"d1cfdece3c511f9058817bb342b8e5c2daa92b3feeae3c1adaf55fe40eb32e9f"} Sep 10 19:54:58.152470 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:58.152399 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2lgfz"] Sep 10 19:54:58.155360 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:54:58.155337 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b74afe_5b35_437d_b27f_2ed325f8e7c8.slice/crio-c84e7478e9affbda953ae6078e5d4b58b632d8afab3cec6f3d5eefc2519a745a WatchSource:0}: Error finding container c84e7478e9affbda953ae6078e5d4b58b632d8afab3cec6f3d5eefc2519a745a: Status 404 returned error can't find the container with id c84e7478e9affbda953ae6078e5d4b58b632d8afab3cec6f3d5eefc2519a745a Sep 10 19:54:59.009719 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:54:59.009679 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2lgfz" event={"ID":"99b74afe-5b35-437d-b27f-2ed325f8e7c8","Type":"ContainerStarted","Data":"c84e7478e9affbda953ae6078e5d4b58b632d8afab3cec6f3d5eefc2519a745a"} Sep 10 19:55:03.021997 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:03.021956 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flfcj" event={"ID":"c2cd9622-ed11-4e4c-b30a-7d94dc097090","Type":"ContainerStarted","Data":"2abf94cbe20913d86b41ff12876da62ca7e2296b135c6a103dfe192bfd92779e"} Sep 10 19:55:03.023578 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:03.023551 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2lgfz" event={"ID":"99b74afe-5b35-437d-b27f-2ed325f8e7c8","Type":"ContainerStarted","Data":"1ba7835ac43cbd2a77d896e7b57ba2d0590135fe3d14ac5eea0b67c85606344c"} Sep 10 19:55:03.023698 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:03.023682 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:55:03.037717 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:03.037673 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-flfcj" podStartSLOduration=66.022942291 podStartE2EDuration="1m11.037658307s" totalImagesPullingTime="5.014716016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:03.036602207 +0000 UTC m=+71.757549327" watchObservedRunningTime="2026-09-10 19:55:03.037658307 +0000 UTC m=+71.758605469" Sep 10 19:55:03.052495 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:03.052309 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2lgfz" podStartSLOduration=66.348506523 podStartE2EDuration="1m11.05228985s" totalImagesPullingTime="4.703783327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:03.052142273 +0000 UTC m=+71.773089393" watchObservedRunningTime="2026-09-10 19:55:03.05228985 +0000 UTC m=+71.773236974" Sep 10 19:55:07.250271 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.250235 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9"] Sep 10 19:55:07.252438 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.252418 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" Sep 10 19:55:07.254732 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.254707 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 19:55:07.254880 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.254738 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 19:55:07.254880 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.254797 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7mwr8\"" Sep 10 19:55:07.259543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.259521 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw"] Sep 10 19:55:07.261219 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.261204 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.263469 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.263407 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 19:55:07.263469 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.263419 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 19:55:07.263469 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.263408 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 19:55:07.263692 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.263670 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9"] Sep 10 19:55:07.263796 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.263761 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xjkjj\"" Sep 10 19:55:07.264009 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.263994 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 19:55:07.273809 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.273788 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw"] Sep 10 19:55:07.350611 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.350578 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x"] Sep 10 19:55:07.353037 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.353022 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:07.356858 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.356835 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 19:55:07.357467 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.357450 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f8vbk\"" Sep 10 19:55:07.357543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.357489 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 19:55:07.357595 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.357488 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 19:55:07.367160 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.367141 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x"] Sep 10 19:55:07.373321 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.373301 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4d3c460-eb05-4132-b042-b961335105d9-serving-cert\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.373422 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.373341 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ng5p\" (UniqueName: \"kubernetes.io/projected/acdd628d-de21-405b-9683-b0b6cc4de4d3-kube-api-access-7ng5p\") pod \"volume-data-source-validator-695c895bd6-mm2w9\" (UID: \"acdd628d-de21-405b-9683-b0b6cc4de4d3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" Sep 10 19:55:07.373422 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.373370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4d3c460-eb05-4132-b042-b961335105d9-config\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.373518 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.373427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvvt\" (UniqueName: \"kubernetes.io/projected/d4d3c460-eb05-4132-b042-b961335105d9-kube-api-access-hwvvt\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.473983 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.473942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ng5p\" (UniqueName: \"kubernetes.io/projected/acdd628d-de21-405b-9683-b0b6cc4de4d3-kube-api-access-7ng5p\") pod \"volume-data-source-validator-695c895bd6-mm2w9\" (UID: \"acdd628d-de21-405b-9683-b0b6cc4de4d3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" Sep 10 19:55:07.473983 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.473982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4d3c460-eb05-4132-b042-b961335105d9-config\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.474200 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.474009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bk4r\" (UniqueName: \"kubernetes.io/projected/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-kube-api-access-6bk4r\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:07.474200 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.474043 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvvt\" (UniqueName: \"kubernetes.io/projected/d4d3c460-eb05-4132-b042-b961335105d9-kube-api-access-hwvvt\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.474200 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.474173 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4d3c460-eb05-4132-b042-b961335105d9-serving-cert\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.474295 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.474202 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:07.474626 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.474606 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4d3c460-eb05-4132-b042-b961335105d9-config\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.476285 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.476269 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4d3c460-eb05-4132-b042-b961335105d9-serving-cert\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.481893 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.481865 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvvt\" (UniqueName: \"kubernetes.io/projected/d4d3c460-eb05-4132-b042-b961335105d9-kube-api-access-hwvvt\") pod \"service-ca-operator-859f94c4b-pf2cw\" (UID: \"d4d3c460-eb05-4132-b042-b961335105d9\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.482095 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.482061 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ng5p\" (UniqueName: \"kubernetes.io/projected/acdd628d-de21-405b-9683-b0b6cc4de4d3-kube-api-access-7ng5p\") pod \"volume-data-source-validator-695c895bd6-mm2w9\" (UID: \"acdd628d-de21-405b-9683-b0b6cc4de4d3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" Sep 10 19:55:07.561799 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.561702 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" Sep 10 19:55:07.569964 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.569916 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" Sep 10 19:55:07.575439 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.575379 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:07.575439 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.575431 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bk4r\" (UniqueName: \"kubernetes.io/projected/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-kube-api-access-6bk4r\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:07.575613 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:07.575590 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 19:55:07.575669 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:07.575658 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls podName:7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:08.075641022 +0000 UTC m=+76.796588138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-msq8x" (UID: "7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253") : secret "samples-operator-tls" not found Sep 10 19:55:07.587678 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.587611 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bk4r\" (UniqueName: \"kubernetes.io/projected/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-kube-api-access-6bk4r\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:07.829972 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.829890 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9"] Sep 10 19:55:07.832555 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:07.832523 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdd628d_de21_405b_9683_b0b6cc4de4d3.slice/crio-ad1e4bda6df05d0eb775ec55fdeb75b81d17b5cdd39560c6ecbfcf7941745bb3 WatchSource:0}: Error finding container ad1e4bda6df05d0eb775ec55fdeb75b81d17b5cdd39560c6ecbfcf7941745bb3: Status 404 returned error can't find the container with id ad1e4bda6df05d0eb775ec55fdeb75b81d17b5cdd39560c6ecbfcf7941745bb3 Sep 10 19:55:07.848977 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:07.848952 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw"] Sep 10 19:55:07.851955 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:07.851932 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d3c460_eb05_4132_b042_b961335105d9.slice/crio-be224f77970c43f312c5af0f12ee45b2c1ac5ae1c732e4d61aabd7c90173f8c9 WatchSource:0}: Error finding container be224f77970c43f312c5af0f12ee45b2c1ac5ae1c732e4d61aabd7c90173f8c9: Status 404 returned error can't find the container with id be224f77970c43f312c5af0f12ee45b2c1ac5ae1c732e4d61aabd7c90173f8c9 Sep 10 19:55:08.037416 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.037376 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" event={"ID":"d4d3c460-eb05-4132-b042-b961335105d9","Type":"ContainerStarted","Data":"be224f77970c43f312c5af0f12ee45b2c1ac5ae1c732e4d61aabd7c90173f8c9"} Sep 10 19:55:08.038480 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.038460 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" event={"ID":"acdd628d-de21-405b-9683-b0b6cc4de4d3","Type":"ContainerStarted","Data":"ad1e4bda6df05d0eb775ec55fdeb75b81d17b5cdd39560c6ecbfcf7941745bb3"} Sep 10 19:55:08.079084 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.079025 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:08.079238 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:08.079194 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 19:55:08.079278 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:08.079261 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls podName:7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:09.079245173 +0000 UTC m=+77.800192296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-msq8x" (UID: "7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253") : secret "samples-operator-tls" not found Sep 10 19:55:08.088152 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.088085 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-ckcjv"] Sep 10 19:55:08.090537 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.090522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" Sep 10 19:55:08.092804 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.092786 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wjt6r\"" Sep 10 19:55:08.102850 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.102639 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-ckcjv"] Sep 10 19:55:08.180383 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.180329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksw6z\" (UniqueName: \"kubernetes.io/projected/767b1a57-f83f-4cf8-9228-7a5afad72a4f-kube-api-access-ksw6z\") pod \"network-check-source-685456655f-ckcjv\" (UID: \"767b1a57-f83f-4cf8-9228-7a5afad72a4f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" Sep 10 19:55:08.280762 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.280713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksw6z\" (UniqueName: \"kubernetes.io/projected/767b1a57-f83f-4cf8-9228-7a5afad72a4f-kube-api-access-ksw6z\") pod \"network-check-source-685456655f-ckcjv\" (UID: \"767b1a57-f83f-4cf8-9228-7a5afad72a4f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" Sep 10 19:55:08.289465 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.289437 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksw6z\" (UniqueName: \"kubernetes.io/projected/767b1a57-f83f-4cf8-9228-7a5afad72a4f-kube-api-access-ksw6z\") pod \"network-check-source-685456655f-ckcjv\" (UID: \"767b1a57-f83f-4cf8-9228-7a5afad72a4f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" Sep 10 19:55:08.399215 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.399126 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" Sep 10 19:55:08.680125 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:08.680031 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-ckcjv"] Sep 10 19:55:08.683901 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:08.683869 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod767b1a57_f83f_4cf8_9228_7a5afad72a4f.slice/crio-871cebe8eeabecbcc77ae9519ecb7164ba2b99152524ea95a3b01328179a5155 WatchSource:0}: Error finding container 871cebe8eeabecbcc77ae9519ecb7164ba2b99152524ea95a3b01328179a5155: Status 404 returned error can't find the container with id 871cebe8eeabecbcc77ae9519ecb7164ba2b99152524ea95a3b01328179a5155 Sep 10 19:55:09.042984 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:09.042953 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" event={"ID":"acdd628d-de21-405b-9683-b0b6cc4de4d3","Type":"ContainerStarted","Data":"6fb5b112ece71c5372562c2fc53589a14e1bacd4a3d37900537153ddf93ffbdc"} Sep 10 19:55:09.044552 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:09.044515 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" event={"ID":"767b1a57-f83f-4cf8-9228-7a5afad72a4f","Type":"ContainerStarted","Data":"ea5c520b76b798a77ce5076323704358c16501127a4b67491c42fcb1f633fc87"} Sep 10 19:55:09.044643 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:09.044577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" event={"ID":"767b1a57-f83f-4cf8-9228-7a5afad72a4f","Type":"ContainerStarted","Data":"871cebe8eeabecbcc77ae9519ecb7164ba2b99152524ea95a3b01328179a5155"} Sep 10 19:55:09.059091 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:09.059022 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-mm2w9" podStartSLOduration=0.917868834 podStartE2EDuration="2.059003738s" totalImagesPullingTime="1.141134904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:09.058397855 +0000 UTC m=+77.779344974" watchObservedRunningTime="2026-09-10 19:55:09.059003738 +0000 UTC m=+77.779950860" Sep 10 19:55:09.074452 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:09.074408 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-ckcjv" podStartSLOduration=1.074393398 podStartE2EDuration="1.074393398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:09.072897574 +0000 UTC m=+77.793844695" watchObservedRunningTime="2026-09-10 19:55:09.074393398 +0000 UTC m=+77.795340517" Sep 10 19:55:09.088213 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:09.088184 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:09.088366 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:09.088322 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 19:55:09.088415 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:09.088377 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls podName:7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:11.088362579 +0000 UTC m=+79.809309701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-msq8x" (UID: "7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253") : secret "samples-operator-tls" not found Sep 10 19:55:10.050465 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:10.050420 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" event={"ID":"d4d3c460-eb05-4132-b042-b961335105d9","Type":"ContainerStarted","Data":"8f6862929f15009940b537e6aaf664338a09681d553f7ace7daceb831d60a93b"} Sep 10 19:55:10.067056 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:10.067004 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" podStartSLOduration=1.037301924 podStartE2EDuration="3.066987705s" totalImagesPullingTime="2.029685781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:10.066274538 +0000 UTC m=+78.787221657" watchObservedRunningTime="2026-09-10 19:55:10.066987705 +0000 UTC m=+78.787934823" Sep 10 19:55:11.103326 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.103284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:11.103729 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:11.103428 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 19:55:11.103729 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:11.103483 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls podName:7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:15.103469382 +0000 UTC m=+83.824416503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-msq8x" (UID: "7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253") : secret "samples-operator-tls" not found Sep 10 19:55:11.799717 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.799684 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4hgsm"] Sep 10 19:55:11.801787 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.801761 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:11.804150 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.804128 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 19:55:11.804293 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.804271 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qkwps\"" Sep 10 19:55:11.804983 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.804964 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 19:55:11.805144 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.805129 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 19:55:11.805242 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.805171 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 19:55:11.817270 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.817235 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4hgsm"] Sep 10 19:55:11.908890 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.908856 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53748e6-5b90-4951-8154-cac98ee3d801-data-volume\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:11.908890 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.908890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:11.909120 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.908920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b53748e6-5b90-4951-8154-cac98ee3d801-crio-socket\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:11.909120 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.908985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9lkx\" (UniqueName: \"kubernetes.io/projected/b53748e6-5b90-4951-8154-cac98ee3d801-kube-api-access-m9lkx\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:11.909120 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:11.909041 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010309 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010265 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010479 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010359 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53748e6-5b90-4951-8154-cac98ee3d801-data-volume\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010479 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.010378 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 19:55:12.010479 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010382 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010479 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.010453 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:12.510431957 +0000 UTC m=+81.231379098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : configmap "kube-rbac-proxy" not found Sep 10 19:55:12.010479 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.010468 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 19:55:12.010720 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010484 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b53748e6-5b90-4951-8154-cac98ee3d801-crio-socket\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010720 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.010518 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:12.51050309 +0000 UTC m=+81.231450187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : secret "insights-runtime-extractor-tls" not found Sep 10 19:55:12.010720 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010547 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b53748e6-5b90-4951-8154-cac98ee3d801-crio-socket\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010720 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9lkx\" (UniqueName: \"kubernetes.io/projected/b53748e6-5b90-4951-8154-cac98ee3d801-kube-api-access-m9lkx\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.010720 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.010691 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53748e6-5b90-4951-8154-cac98ee3d801-data-volume\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.022619 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.022594 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9lkx\" (UniqueName: \"kubernetes.io/projected/b53748e6-5b90-4951-8154-cac98ee3d801-kube-api-access-m9lkx\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.515064 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.515025 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.515462 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:12.515106 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:12.515462 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.515161 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 19:55:12.515462 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.515211 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 19:55:12.515462 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.515232 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:13.515214578 +0000 UTC m=+82.236161696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : configmap "kube-rbac-proxy" not found Sep 10 19:55:12.515462 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:12.515251 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:13.515240347 +0000 UTC m=+82.236187469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : secret "insights-runtime-extractor-tls" not found Sep 10 19:55:13.522784 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:13.522746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:13.523320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:13.522828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:13.523320 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:13.522862 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 19:55:13.523320 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:13.522920 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:15.522906801 +0000 UTC m=+84.243853911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : configmap "kube-rbac-proxy" not found Sep 10 19:55:13.523320 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:13.522945 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 19:55:13.523320 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:13.522997 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:15.522979422 +0000 UTC m=+84.243926538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : secret "insights-runtime-extractor-tls" not found Sep 10 19:55:14.763462 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:14.763350 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tpt2k_1a17871d-91b4-42ed-90ae-2feee6eb985a/dns-node-resolver/0.log" Sep 10 19:55:15.133731 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:15.133631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:15.133882 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:15.133789 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 19:55:15.133882 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:15.133851 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls podName:7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:23.133836501 +0000 UTC m=+91.854783598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-msq8x" (UID: "7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253") : secret "samples-operator-tls" not found Sep 10 19:55:15.363420 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:15.363388 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dspv5_1c2a39fa-8215-4571-aa00-cdcd08fc3087/node-ca/0.log" Sep 10 19:55:15.536874 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:15.536841 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:15.537059 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:15.536907 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:15.537059 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:15.536999 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 19:55:15.537059 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:15.537019 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 19:55:15.537222 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:15.537095 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:19.5370559 +0000 UTC m=+88.258003563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : configmap "kube-rbac-proxy" not found Sep 10 19:55:15.537222 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:15.537119 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:19.537109331 +0000 UTC m=+88.258056428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : secret "insights-runtime-extractor-tls" not found Sep 10 19:55:16.162883 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:16.162852 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-glzkt_a0a7eead-c04c-4105-82b5-d060fe76f2ef/serve-healthcheck-canary/0.log" Sep 10 19:55:19.567187 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:19.567152 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:19.567586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:19.567252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:19.567586 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:19.567296 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 19:55:19.567586 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:19.567331 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 19:55:19.567586 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:19.567372 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:27.567356097 +0000 UTC m=+96.288303195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : secret "insights-runtime-extractor-tls" not found Sep 10 19:55:19.567586 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:19.567437 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:27.567418843 +0000 UTC m=+96.288365978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : configmap "kube-rbac-proxy" not found Sep 10 19:55:23.196465 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:23.196420 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:23.198841 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:23.198820 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-msq8x\" (UID: \"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:23.261985 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:23.261937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" Sep 10 19:55:23.531447 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:23.531422 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x"] Sep 10 19:55:24.099446 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:24.099408 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" event={"ID":"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253","Type":"ContainerStarted","Data":"5349db81f5f869e480682d3d6e6a209587ef4b05d959db79ea9b0539e736ced2"} Sep 10 19:55:26.107407 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:26.107372 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" event={"ID":"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253","Type":"ContainerStarted","Data":"755a8952e70db00350c963c254df92f74ea7e49004e34a4bd007e923a331ab7c"} Sep 10 19:55:26.107813 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:26.107423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" event={"ID":"7f4f2a8c-5ca0-40f5-ad52-2b4c38e03253","Type":"ContainerStarted","Data":"6d301862bf5f6e3a9d940dfe2f6d6c3cd499745ba03f1110833b22d122e93646"} Sep 10 19:55:26.139315 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:26.139264 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-msq8x" podStartSLOduration=17.562708481 podStartE2EDuration="19.139247814s" totalImagesPullingTime="1.576539333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:26.137773219 +0000 UTC m=+94.858720338" watchObservedRunningTime="2026-09-10 19:55:26.139247814 +0000 UTC m=+94.860194933" Sep 10 19:55:27.629583 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:27.629524 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:27.629981 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:27.629611 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:27.629981 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:27.629725 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 19:55:27.629981 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:27.629792 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls podName:b53748e6-5b90-4951-8154-cac98ee3d801 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:43.629774313 +0000 UTC m=+112.350721409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4hgsm" (UID: "b53748e6-5b90-4951-8154-cac98ee3d801") : secret "insights-runtime-extractor-tls" not found Sep 10 19:55:27.630245 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:27.630226 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b53748e6-5b90-4951-8154-cac98ee3d801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:28.535573 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.535541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:55:28.535803 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.535583 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:55:28.537913 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.537886 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e92499b-cb2e-4203-8247-f3ca5d908d4e-metrics-tls\") pod \"dns-default-tfpxv\" (UID: \"6e92499b-cb2e-4203-8247-f3ca5d908d4e\") " pod="openshift-dns/dns-default-tfpxv" Sep 10 19:55:28.538017 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.537999 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"image-registry-6ff7548c8d-d2kxn\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:55:28.562174 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.562146 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ngfwp\"" Sep 10 19:55:28.569575 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.569556 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfpxv" Sep 10 19:55:28.838134 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.838038 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2xjwm\"" Sep 10 19:55:28.845440 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.845417 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:55:28.862668 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:28.862638 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tfpxv"] Sep 10 19:55:28.864442 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:28.864419 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e92499b_cb2e_4203_8247_f3ca5d908d4e.slice/crio-461801c3f49ed3e1d5cb6c1d8bce69abf67b4ee8e887e3c074f2bb341ef93f22 WatchSource:0}: Error finding container 461801c3f49ed3e1d5cb6c1d8bce69abf67b4ee8e887e3c074f2bb341ef93f22: Status 404 returned error can't find the container with id 461801c3f49ed3e1d5cb6c1d8bce69abf67b4ee8e887e3c074f2bb341ef93f22 Sep 10 19:55:29.117883 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:29.117804 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfpxv" event={"ID":"6e92499b-cb2e-4203-8247-f3ca5d908d4e","Type":"ContainerStarted","Data":"461801c3f49ed3e1d5cb6c1d8bce69abf67b4ee8e887e3c074f2bb341ef93f22"} Sep 10 19:55:29.123945 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:29.123919 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ff7548c8d-d2kxn"] Sep 10 19:55:29.126566 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:29.126547 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f26794b_b02c_4870_9cd2_0be56dfb642a.slice/crio-e55fc6b4a0c1ba8efa4b17840e803334f5f59ac0324557eb303d7ded4a5883c4 WatchSource:0}: Error finding container e55fc6b4a0c1ba8efa4b17840e803334f5f59ac0324557eb303d7ded4a5883c4: Status 404 returned error can't find the container with id e55fc6b4a0c1ba8efa4b17840e803334f5f59ac0324557eb303d7ded4a5883c4 Sep 10 19:55:30.123443 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:30.123405 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" event={"ID":"8f26794b-b02c-4870-9cd2-0be56dfb642a","Type":"ContainerStarted","Data":"a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e"} Sep 10 19:55:30.123942 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:30.123464 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" event={"ID":"8f26794b-b02c-4870-9cd2-0be56dfb642a","Type":"ContainerStarted","Data":"e55fc6b4a0c1ba8efa4b17840e803334f5f59ac0324557eb303d7ded4a5883c4"} Sep 10 19:55:30.123942 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:30.123523 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:55:30.156197 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:30.156144 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" podStartSLOduration=98.156127113 podStartE2EDuration="1m38.156127113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:30.155645887 +0000 UTC m=+98.876593004" watchObservedRunningTime="2026-09-10 19:55:30.156127113 +0000 UTC m=+98.877074231" Sep 10 19:55:31.128990 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:31.128952 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfpxv" event={"ID":"6e92499b-cb2e-4203-8247-f3ca5d908d4e","Type":"ContainerStarted","Data":"6709e50e8b79a49941e6a542f541f6540825768d7d31405ed8f62b0e0f12eb14"} Sep 10 19:55:31.129493 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:31.129012 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfpxv" event={"ID":"6e92499b-cb2e-4203-8247-f3ca5d908d4e","Type":"ContainerStarted","Data":"ee1a5d7ddda938cd55e6c558dfb0c89f0c769ed01cb3969798d703cade94236d"} Sep 10 19:55:31.129493 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:31.129332 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tfpxv" Sep 10 19:55:31.148472 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:31.148433 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tfpxv" podStartSLOduration=65.498897982 podStartE2EDuration="1m7.148420725s" totalImagesPullingTime="1.649522743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:31.147254109 +0000 UTC m=+99.868201232" watchObservedRunningTime="2026-09-10 19:55:31.148420725 +0000 UTC m=+99.869367896" Sep 10 19:55:34.028320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:34.028290 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2lgfz" Sep 10 19:55:37.637468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.637437 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-6l6vv"] Sep 10 19:55:37.640343 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.640327 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.643266 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.643244 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 19:55:37.644767 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.644746 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7zxf7\"" Sep 10 19:55:37.644883 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.644768 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 19:55:37.649158 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.649137 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh"] Sep 10 19:55:37.650935 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.650918 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.657751 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.657733 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 19:55:37.657837 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.657771 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 19:55:37.663040 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.663017 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-6l6vv"] Sep 10 19:55:37.663894 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.663879 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 19:55:37.664040 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.664024 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 19:55:37.689298 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.689275 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh"] Sep 10 19:55:37.764474 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.764446 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-5w6xl"] Sep 10 19:55:37.766540 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.766519 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:37.768976 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.768954 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 19:55:37.768976 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.768976 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9bwh7\"" Sep 10 19:55:37.769171 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.768991 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 19:55:37.787925 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.787902 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-5w6xl"] Sep 10 19:55:37.799931 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.799909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/84bd2c76-eb69-4f97-9662-b8b1272599f8-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-6l6vv\" (UID: \"84bd2c76-eb69-4f97-9662-b8b1272599f8\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.800042 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.799946 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/84bd2c76-eb69-4f97-9662-b8b1272599f8-nginx-conf\") pod \"networking-console-plugin-79977f9844-6l6vv\" (UID: \"84bd2c76-eb69-4f97-9662-b8b1272599f8\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.800042 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.799967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/364a7021-682b-4896-a1ca-c752f041a59b-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.800042 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.800012 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hk9q\" (UniqueName: \"kubernetes.io/projected/364a7021-682b-4896-a1ca-c752f041a59b-kube-api-access-8hk9q\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.800170 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.800044 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364a7021-682b-4896-a1ca-c752f041a59b-tmp\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.900918 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.900807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/84bd2c76-eb69-4f97-9662-b8b1272599f8-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-6l6vv\" (UID: \"84bd2c76-eb69-4f97-9662-b8b1272599f8\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.900918 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.900889 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/84bd2c76-eb69-4f97-9662-b8b1272599f8-nginx-conf\") pod \"networking-console-plugin-79977f9844-6l6vv\" (UID: \"84bd2c76-eb69-4f97-9662-b8b1272599f8\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.900918 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.900917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/364a7021-682b-4896-a1ca-c752f041a59b-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.901205 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.900947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hk9q\" (UniqueName: \"kubernetes.io/projected/364a7021-682b-4896-a1ca-c752f041a59b-kube-api-access-8hk9q\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.901205 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.900974 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364a7021-682b-4896-a1ca-c752f041a59b-tmp\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.901205 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.901018 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csh7b\" (UniqueName: \"kubernetes.io/projected/6f7f7aff-7891-47be-b85a-0c30ead85c10-kube-api-access-csh7b\") pod \"downloads-85d98c998f-5w6xl\" (UID: \"6f7f7aff-7891-47be-b85a-0c30ead85c10\") " pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:37.901639 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.901517 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364a7021-682b-4896-a1ca-c752f041a59b-tmp\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.903458 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.903431 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/84bd2c76-eb69-4f97-9662-b8b1272599f8-nginx-conf\") pod \"networking-console-plugin-79977f9844-6l6vv\" (UID: \"84bd2c76-eb69-4f97-9662-b8b1272599f8\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.903938 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.903917 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/364a7021-682b-4896-a1ca-c752f041a59b-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.904157 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.904133 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/84bd2c76-eb69-4f97-9662-b8b1272599f8-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-6l6vv\" (UID: \"84bd2c76-eb69-4f97-9662-b8b1272599f8\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.911708 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.911679 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hk9q\" (UniqueName: \"kubernetes.io/projected/364a7021-682b-4896-a1ca-c752f041a59b-kube-api-access-8hk9q\") pod \"klusterlet-addon-workmgr-6d9f56fd5d-x6lsh\" (UID: \"364a7021-682b-4896-a1ca-c752f041a59b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:37.949665 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.949632 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" Sep 10 19:55:37.960771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:37.960744 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:38.001898 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:38.001816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csh7b\" (UniqueName: \"kubernetes.io/projected/6f7f7aff-7891-47be-b85a-0c30ead85c10-kube-api-access-csh7b\") pod \"downloads-85d98c998f-5w6xl\" (UID: \"6f7f7aff-7891-47be-b85a-0c30ead85c10\") " pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:38.015116 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:38.015060 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csh7b\" (UniqueName: \"kubernetes.io/projected/6f7f7aff-7891-47be-b85a-0c30ead85c10-kube-api-access-csh7b\") pod \"downloads-85d98c998f-5w6xl\" (UID: \"6f7f7aff-7891-47be-b85a-0c30ead85c10\") " pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:38.075237 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:38.075207 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:38.221035 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:38.221007 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-6l6vv"] Sep 10 19:55:38.223361 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:38.223335 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84bd2c76_eb69_4f97_9662_b8b1272599f8.slice/crio-adbb13cd1c356b15fec12869f665aa538c5d271815fad608361a9a0e30b149ba WatchSource:0}: Error finding container adbb13cd1c356b15fec12869f665aa538c5d271815fad608361a9a0e30b149ba: Status 404 returned error can't find the container with id adbb13cd1c356b15fec12869f665aa538c5d271815fad608361a9a0e30b149ba Sep 10 19:55:38.250880 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:38.250826 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh"] Sep 10 19:55:38.253272 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:38.253251 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364a7021_682b_4896_a1ca_c752f041a59b.slice/crio-c16b41e6d5b2b1d0a14b5342a5f33d85efce5cfcb133c819b923532b1b36228d WatchSource:0}: Error finding container c16b41e6d5b2b1d0a14b5342a5f33d85efce5cfcb133c819b923532b1b36228d: Status 404 returned error can't find the container with id c16b41e6d5b2b1d0a14b5342a5f33d85efce5cfcb133c819b923532b1b36228d Sep 10 19:55:38.341629 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:38.341605 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-5w6xl"] Sep 10 19:55:38.343855 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:38.343834 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7f7aff_7891_47be_b85a_0c30ead85c10.slice/crio-3af72e17de6e0e57e383f4ac7d63134719366a7b0a4ea545c6dc1f0aa6ca82bd WatchSource:0}: Error finding container 3af72e17de6e0e57e383f4ac7d63134719366a7b0a4ea545c6dc1f0aa6ca82bd: Status 404 returned error can't find the container with id 3af72e17de6e0e57e383f4ac7d63134719366a7b0a4ea545c6dc1f0aa6ca82bd Sep 10 19:55:39.164209 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:39.164170 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" event={"ID":"364a7021-682b-4896-a1ca-c752f041a59b","Type":"ContainerStarted","Data":"c16b41e6d5b2b1d0a14b5342a5f33d85efce5cfcb133c819b923532b1b36228d"} Sep 10 19:55:39.167769 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:39.167735 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" event={"ID":"84bd2c76-eb69-4f97-9662-b8b1272599f8","Type":"ContainerStarted","Data":"adbb13cd1c356b15fec12869f665aa538c5d271815fad608361a9a0e30b149ba"} Sep 10 19:55:39.171338 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:39.171278 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-5w6xl" event={"ID":"6f7f7aff-7891-47be-b85a-0c30ead85c10","Type":"ContainerStarted","Data":"3af72e17de6e0e57e383f4ac7d63134719366a7b0a4ea545c6dc1f0aa6ca82bd"} Sep 10 19:55:41.138209 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:41.138168 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tfpxv" Sep 10 19:55:41.182905 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:41.182786 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" event={"ID":"84bd2c76-eb69-4f97-9662-b8b1272599f8","Type":"ContainerStarted","Data":"58fad4cc2e7421d04a0390cc608501dcda16671b7a5a8c586546ae045bf41c25"} Sep 10 19:55:41.203824 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:41.203767 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-6l6vv" podStartSLOduration=1.526615539 podStartE2EDuration="4.203746297s" totalImagesPullingTime="2.677130758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:41.202039533 +0000 UTC m=+109.922986684" watchObservedRunningTime="2026-09-10 19:55:41.203746297 +0000 UTC m=+109.924693419" Sep 10 19:55:42.347433 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.347402 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7456bc86c6-g455v"] Sep 10 19:55:42.350537 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.350477 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.354314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.354259 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 19:55:42.354314 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.354261 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 19:55:42.354528 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.354319 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 19:55:42.354528 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.354286 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-t4bpf\"" Sep 10 19:55:42.354643 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.354540 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 19:55:42.354643 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.354561 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 19:55:42.361773 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.361724 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7456bc86c6-g455v"] Sep 10 19:55:42.441833 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.441810 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgmf6\" (UniqueName: \"kubernetes.io/projected/4a4d39c9-8a77-415f-85f8-2af94596c0ed-kube-api-access-jgmf6\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.441936 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.441843 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-serving-cert\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.441936 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.441874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-config\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.441936 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.441905 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-oauth-serving-cert\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.441936 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.441931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-service-ca\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.442143 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.441964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-oauth-config\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.543362 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.543325 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgmf6\" (UniqueName: \"kubernetes.io/projected/4a4d39c9-8a77-415f-85f8-2af94596c0ed-kube-api-access-jgmf6\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.543541 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.543379 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-serving-cert\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.543541 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.543416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-config\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.543541 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.543443 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-oauth-serving-cert\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.543541 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.543467 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-service-ca\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.543756 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.543539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-oauth-config\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.544322 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.544296 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-service-ca\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.544439 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.544380 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-oauth-serving-cert\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.544439 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.544382 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-config\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.546766 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.546720 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-oauth-config\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.547060 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.547040 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-serving-cert\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.566585 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.566553 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgmf6\" (UniqueName: \"kubernetes.io/projected/4a4d39c9-8a77-415f-85f8-2af94596c0ed-kube-api-access-jgmf6\") pod \"console-7456bc86c6-g455v\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.662095 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.661997 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:42.884788 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.884753 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6"] Sep 10 19:55:42.888063 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.888042 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:42.890957 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.890936 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 19:55:42.891153 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.891133 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vr5x8\"" Sep 10 19:55:42.907324 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.907295 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6"] Sep 10 19:55:42.954315 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:42.954286 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7456bc86c6-g455v"] Sep 10 19:55:42.956886 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:42.956860 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4d39c9_8a77_415f_85f8_2af94596c0ed.slice/crio-c096d546275b11f1e73e1f8d2e0373706105a368474009530ed35be0c2bca83a WatchSource:0}: Error finding container c096d546275b11f1e73e1f8d2e0373706105a368474009530ed35be0c2bca83a: Status 404 returned error can't find the container with id c096d546275b11f1e73e1f8d2e0373706105a368474009530ed35be0c2bca83a Sep 10 19:55:43.048627 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.048525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a236573e-f358-4fdf-aba3-f3b067e5473e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-g9zg6\" (UID: \"a236573e-f358-4fdf-aba3-f3b067e5473e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:43.150569 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.150536 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a236573e-f358-4fdf-aba3-f3b067e5473e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-g9zg6\" (UID: \"a236573e-f358-4fdf-aba3-f3b067e5473e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:43.153320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.153294 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a236573e-f358-4fdf-aba3-f3b067e5473e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-g9zg6\" (UID: \"a236573e-f358-4fdf-aba3-f3b067e5473e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:43.191240 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.191201 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456bc86c6-g455v" event={"ID":"4a4d39c9-8a77-415f-85f8-2af94596c0ed","Type":"ContainerStarted","Data":"c096d546275b11f1e73e1f8d2e0373706105a368474009530ed35be0c2bca83a"} Sep 10 19:55:43.193047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.193012 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" event={"ID":"364a7021-682b-4896-a1ca-c752f041a59b","Type":"ContainerStarted","Data":"a29ef2fd899533de4f5fb82da53d57f3feaedb78dc31b550335a0938802953ab"} Sep 10 19:55:43.193591 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.193569 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:43.195661 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.195641 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" Sep 10 19:55:43.198777 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.198747 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:43.210501 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.210434 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d9f56fd5d-x6lsh" podStartSLOduration=2.044935079 podStartE2EDuration="6.210406137s" totalImagesPullingTime="4.165471058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:43.209701976 +0000 UTC m=+111.930649108" watchObservedRunningTime="2026-09-10 19:55:43.210406137 +0000 UTC m=+111.931353248" Sep 10 19:55:43.486233 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.486157 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6"] Sep 10 19:55:43.489374 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:43.489344 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda236573e_f358_4fdf_aba3_f3b067e5473e.slice/crio-540947da6f39499a39e92ac996a139ae6a1f2d1cb12b8fddd4fe44fc00ce3cc5 WatchSource:0}: Error finding container 540947da6f39499a39e92ac996a139ae6a1f2d1cb12b8fddd4fe44fc00ce3cc5: Status 404 returned error can't find the container with id 540947da6f39499a39e92ac996a139ae6a1f2d1cb12b8fddd4fe44fc00ce3cc5 Sep 10 19:55:43.655877 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.655847 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:43.658385 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.658360 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b53748e6-5b90-4951-8154-cac98ee3d801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4hgsm\" (UID: \"b53748e6-5b90-4951-8154-cac98ee3d801\") " pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:43.910786 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:43.910752 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4hgsm" Sep 10 19:55:44.201675 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:44.201595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4hgsm"] Sep 10 19:55:44.202823 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:44.201891 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" event={"ID":"a236573e-f358-4fdf-aba3-f3b067e5473e","Type":"ContainerStarted","Data":"540947da6f39499a39e92ac996a139ae6a1f2d1cb12b8fddd4fe44fc00ce3cc5"} Sep 10 19:55:44.206640 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:44.206616 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53748e6_5b90_4951_8154_cac98ee3d801.slice/crio-3d8ac2b9397e615d5d95590b17ee52e0e6471465cd4609ab82fdbbb0a0ab98db WatchSource:0}: Error finding container 3d8ac2b9397e615d5d95590b17ee52e0e6471465cd4609ab82fdbbb0a0ab98db: Status 404 returned error can't find the container with id 3d8ac2b9397e615d5d95590b17ee52e0e6471465cd4609ab82fdbbb0a0ab98db Sep 10 19:55:45.208192 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.208155 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" event={"ID":"a236573e-f358-4fdf-aba3-f3b067e5473e","Type":"ContainerStarted","Data":"074263f8d6bcb8a97e571b1d6feca6e86c747694b46e0de5876b6b6ad3c635a1"} Sep 10 19:55:45.208886 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.208698 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:45.212872 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.212667 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4hgsm" event={"ID":"b53748e6-5b90-4951-8154-cac98ee3d801","Type":"ContainerStarted","Data":"5ec11ea3e0c3d91ab0c8115a21360185b98c890d4464b4cc7e1fe13276228ef3"} Sep 10 19:55:45.212872 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.212725 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4hgsm" event={"ID":"b53748e6-5b90-4951-8154-cac98ee3d801","Type":"ContainerStarted","Data":"e6c217fb2715e6d99e258711602ebd28248a75a79b37f450a38b622312781e33"} Sep 10 19:55:45.212872 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.212760 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4hgsm" event={"ID":"b53748e6-5b90-4951-8154-cac98ee3d801","Type":"ContainerStarted","Data":"3d8ac2b9397e615d5d95590b17ee52e0e6471465cd4609ab82fdbbb0a0ab98db"} Sep 10 19:55:45.215630 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.215606 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" Sep 10 19:55:45.227160 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:45.227112 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-g9zg6" podStartSLOduration=1.880720707 podStartE2EDuration="3.227094574s" totalImagesPullingTime="1.346373867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:45.226118763 +0000 UTC m=+113.947065878" watchObservedRunningTime="2026-09-10 19:55:45.227094574 +0000 UTC m=+113.948041688" Sep 10 19:55:47.223315 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:47.223272 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456bc86c6-g455v" event={"ID":"4a4d39c9-8a77-415f-85f8-2af94596c0ed","Type":"ContainerStarted","Data":"534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4"} Sep 10 19:55:47.245731 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:47.245625 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7456bc86c6-g455v" podStartSLOduration=1.587712818 podStartE2EDuration="5.24560732s" totalImagesPullingTime="3.657894502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:47.243775897 +0000 UTC m=+115.964723020" watchObservedRunningTime="2026-09-10 19:55:47.24560732 +0000 UTC m=+115.966554439" Sep 10 19:55:48.229283 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:48.229244 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4hgsm" event={"ID":"b53748e6-5b90-4951-8154-cac98ee3d801","Type":"ContainerStarted","Data":"0273399eba62ffe2afeb8e1139c26fee0160632f7c8fb5511934ab79c36117b1"} Sep 10 19:55:48.248259 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:48.248213 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4hgsm" podStartSLOduration=33.980551346 podStartE2EDuration="37.248192287s" totalImagesPullingTime="3.267640941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:55:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:48.246786102 +0000 UTC m=+116.967733222" watchObservedRunningTime="2026-09-10 19:55:48.248192287 +0000 UTC m=+116.969139405" Sep 10 19:55:51.134012 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.133977 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:55:51.329344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.329312 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh"] Sep 10 19:55:51.333961 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.333937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.336491 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.336469 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 19:55:51.336608 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.336516 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 19:55:51.336608 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.336532 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 19:55:51.336894 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.336847 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 19:55:51.336894 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.336847 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hnq9j\"" Sep 10 19:55:51.337066 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.336959 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 19:55:51.344632 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.344611 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-d248x"] Sep 10 19:55:51.347722 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.347703 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7xgb9"] Sep 10 19:55:51.347908 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.347885 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.351290 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.351213 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh"] Sep 10 19:55:51.351382 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.351329 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.351470 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.351450 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 19:55:51.351527 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.351480 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 19:55:51.351859 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.351717 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 19:55:51.351859 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.351814 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hgzpm\"" Sep 10 19:55:51.354338 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.354321 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 19:55:51.354906 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.354891 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 19:55:51.355108 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.355058 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d8qrq\"" Sep 10 19:55:51.355305 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.355290 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 19:55:51.365769 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.365746 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-d248x"] Sep 10 19:55:51.527805 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.527772 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-wtmp\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.527805 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.527806 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.528047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.527836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.528047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.527861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqcfd\" (UniqueName: \"kubernetes.io/projected/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-api-access-sqcfd\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.528047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.527921 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgwqw\" (UniqueName: \"kubernetes.io/projected/1c955aee-1ab4-474e-86c3-bd3f46776c8e-kube-api-access-bgwqw\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.527990 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528237 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528083 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-root\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528237 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528126 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-textfile\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528237 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528153 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d701640-ef43-4707-8982-eb3aceaba4b6-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.528237 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.528237 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528207 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d701640-ef43-4707-8982-eb3aceaba4b6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.528468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528239 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6d9r\" (UniqueName: \"kubernetes.io/projected/5d701640-ef43-4707-8982-eb3aceaba4b6-kube-api-access-j6d9r\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.528468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528263 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-sys\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d701640-ef43-4707-8982-eb3aceaba4b6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.528468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528355 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.528468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528451 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528635 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528481 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c955aee-1ab4-474e-86c3-bd3f46776c8e-metrics-client-ca\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528635 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528498 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-tls\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.528635 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.528551 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.629344 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-wtmp\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629355 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.629514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629386 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.629514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629415 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqcfd\" (UniqueName: \"kubernetes.io/projected/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-api-access-sqcfd\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.629514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgwqw\" (UniqueName: \"kubernetes.io/projected/1c955aee-1ab4-474e-86c3-bd3f46776c8e-kube-api-access-bgwqw\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629514 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629479 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-wtmp\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629536 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-root\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629587 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-textfile\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629596 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-root\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629617 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d701640-ef43-4707-8982-eb3aceaba4b6-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629646 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629680 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d701640-ef43-4707-8982-eb3aceaba4b6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629714 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6d9r\" (UniqueName: \"kubernetes.io/projected/5d701640-ef43-4707-8982-eb3aceaba4b6-kube-api-access-j6d9r\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-sys\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.629774 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d701640-ef43-4707-8982-eb3aceaba4b6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.630276 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.630276 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.630276 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c955aee-1ab4-474e-86c3-bd3f46776c8e-metrics-client-ca\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.630276 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629894 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-tls\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.630276 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.629932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.630510 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.630441 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.630673 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.630590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.630673 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.630602 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-textfile\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.630673 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.630655 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c955aee-1ab4-474e-86c3-bd3f46776c8e-sys\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.631121 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.631096 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.631212 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:51.631202 2570 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 10 19:55:51.631279 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.631221 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.631279 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.631233 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c955aee-1ab4-474e-86c3-bd3f46776c8e-metrics-client-ca\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.631279 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:51.631262 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-tls podName:107b8bf9-9f05-469b-ac9c-9dc8e7b24f28 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:52.131240402 +0000 UTC m=+120.852187513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-d248x" (UID: "107b8bf9-9f05-469b-ac9c-9dc8e7b24f28") : secret "kube-state-metrics-tls" not found Sep 10 19:55:51.631951 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.631881 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d701640-ef43-4707-8982-eb3aceaba4b6-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.633736 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.633710 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.633818 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.633685 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d701640-ef43-4707-8982-eb3aceaba4b6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.633968 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.633938 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.634092 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.634049 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d701640-ef43-4707-8982-eb3aceaba4b6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.634494 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.634470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c955aee-1ab4-474e-86c3-bd3f46776c8e-node-exporter-tls\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.644049 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.644024 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgwqw\" (UniqueName: \"kubernetes.io/projected/1c955aee-1ab4-474e-86c3-bd3f46776c8e-kube-api-access-bgwqw\") pod \"node-exporter-7xgb9\" (UID: \"1c955aee-1ab4-474e-86c3-bd3f46776c8e\") " pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:51.644192 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.644033 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqcfd\" (UniqueName: \"kubernetes.io/projected/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-api-access-sqcfd\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:51.644431 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.644410 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6d9r\" (UniqueName: \"kubernetes.io/projected/5d701640-ef43-4707-8982-eb3aceaba4b6-kube-api-access-j6d9r\") pod \"openshift-state-metrics-657fd6c54-tmsrh\" (UID: \"5d701640-ef43-4707-8982-eb3aceaba4b6\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.645062 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.645029 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" Sep 10 19:55:51.671367 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.671340 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d8qrq\"" Sep 10 19:55:51.679377 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:51.679351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7xgb9" Sep 10 19:55:52.135231 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.135192 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:52.138379 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.138350 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/107b8bf9-9f05-469b-ac9c-9dc8e7b24f28-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d248x\" (UID: \"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:52.263581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.263548 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hgzpm\"" Sep 10 19:55:52.271640 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.271613 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" Sep 10 19:55:52.524832 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.524799 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 19:55:52.529252 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.529230 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.532017 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.531993 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 19:55:52.532148 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532026 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 19:55:52.532148 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532001 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 19:55:52.532281 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532201 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 19:55:52.532345 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532295 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 19:55:52.532394 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532351 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 19:55:52.532677 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532655 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6msvl\"" Sep 10 19:55:52.532772 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.532715 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 19:55:52.533317 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.533299 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 19:55:52.533418 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.533399 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 19:55:52.550958 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.550934 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 19:55:52.639632 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639596 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5fd52e1-96bc-4169-84c5-a61d15d40e04-config-out\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.639632 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639641 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.639853 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639702 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.639853 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639742 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-config-volume\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.639853 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639756 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5fd52e1-96bc-4169-84c5-a61d15d40e04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.639853 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639853 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5fd52e1-96bc-4169-84c5-a61d15d40e04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639877 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlq57\" (UniqueName: \"kubernetes.io/projected/e5fd52e1-96bc-4169-84c5-a61d15d40e04-kube-api-access-vlq57\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5fd52e1-96bc-4169-84c5-a61d15d40e04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-web-config\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639957 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5fd52e1-96bc-4169-84c5-a61d15d40e04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.639989 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.640052 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.640014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.662637 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.662607 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:52.662819 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.662695 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:52.668318 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.668293 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:52.740661 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-config-volume\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.740661 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5fd52e1-96bc-4169-84c5-a61d15d40e04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.740907 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740718 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.740907 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740744 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5fd52e1-96bc-4169-84c5-a61d15d40e04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.740907 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlq57\" (UniqueName: \"kubernetes.io/projected/e5fd52e1-96bc-4169-84c5-a61d15d40e04-kube-api-access-vlq57\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5fd52e1-96bc-4169-84c5-a61d15d40e04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.740983 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-web-config\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741019 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5fd52e1-96bc-4169-84c5-a61d15d40e04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741103 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741167 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5fd52e1-96bc-4169-84c5-a61d15d40e04-config-out\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741195 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741866 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741690 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5fd52e1-96bc-4169-84c5-a61d15d40e04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.741866 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:52.741698 2570 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 10 19:55:52.741866 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:52.741774 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-main-tls podName:e5fd52e1-96bc-4169-84c5-a61d15d40e04 nodeName:}" failed. No retries permitted until 2026-09-10 19:55:53.241754793 +0000 UTC m=+121.962701906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "e5fd52e1-96bc-4169-84c5-a61d15d40e04") : secret "alertmanager-main-tls" not found Sep 10 19:55:52.742013 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.741935 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5fd52e1-96bc-4169-84c5-a61d15d40e04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.742572 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.742546 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5fd52e1-96bc-4169-84c5-a61d15d40e04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.743947 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.743904 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.744048 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.743948 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5fd52e1-96bc-4169-84c5-a61d15d40e04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.744383 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.744359 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-config-volume\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.744468 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.744421 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.744849 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.744806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.745640 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.745618 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-web-config\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.745872 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.745822 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5fd52e1-96bc-4169-84c5-a61d15d40e04-config-out\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.746619 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.746582 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:52.775607 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:52.775524 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlq57\" (UniqueName: \"kubernetes.io/projected/e5fd52e1-96bc-4169-84c5-a61d15d40e04-kube-api-access-vlq57\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:53.246415 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:53.246383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:53.249520 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:53.249487 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5fd52e1-96bc-4169-84c5-a61d15d40e04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5fd52e1-96bc-4169-84c5-a61d15d40e04\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:53.254122 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:53.254101 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:55:53.441476 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:53.441444 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 19:55:55.615487 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:55.615288 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c955aee_1ab4_474e_86c3_bd3f46776c8e.slice/crio-0696f93a1e95b023173cc4aab9fefebff85c842ffacd7fc9956a15de36182a0f WatchSource:0}: Error finding container 0696f93a1e95b023173cc4aab9fefebff85c842ffacd7fc9956a15de36182a0f: Status 404 returned error can't find the container with id 0696f93a1e95b023173cc4aab9fefebff85c842ffacd7fc9956a15de36182a0f Sep 10 19:55:55.889305 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:55.889271 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-d248x"] Sep 10 19:55:55.891982 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:55.891953 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod107b8bf9_9f05_469b_ac9c_9dc8e7b24f28.slice/crio-6bf5a665a4986fe987609d9ae59f8a75f2aa36f047078d154789d1492e943cf0 WatchSource:0}: Error finding container 6bf5a665a4986fe987609d9ae59f8a75f2aa36f047078d154789d1492e943cf0: Status 404 returned error can't find the container with id 6bf5a665a4986fe987609d9ae59f8a75f2aa36f047078d154789d1492e943cf0 Sep 10 19:55:55.902003 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:55.901975 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh"] Sep 10 19:55:55.904874 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:55.903987 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d701640_ef43_4707_8982_eb3aceaba4b6.slice/crio-b345927a39e96fa9f9394bf95910bd0c6a1647ab9e1cdf1b55166488dafe740b WatchSource:0}: Error finding container b345927a39e96fa9f9394bf95910bd0c6a1647ab9e1cdf1b55166488dafe740b: Status 404 returned error can't find the container with id b345927a39e96fa9f9394bf95910bd0c6a1647ab9e1cdf1b55166488dafe740b Sep 10 19:55:55.906392 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:55.906367 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 19:55:55.908750 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:55.908726 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5fd52e1_96bc_4169_84c5_a61d15d40e04.slice/crio-941a86b8dbeeca21339111c822aa0fbde814199e01a10d956a9c14fbc8206b09 WatchSource:0}: Error finding container 941a86b8dbeeca21339111c822aa0fbde814199e01a10d956a9c14fbc8206b09: Status 404 returned error can't find the container with id 941a86b8dbeeca21339111c822aa0fbde814199e01a10d956a9c14fbc8206b09 Sep 10 19:55:56.110508 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.110473 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr"] Sep 10 19:55:56.115039 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.114909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.118627 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.118602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 19:55:56.118877 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.118626 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2cwn4\"" Sep 10 19:55:56.128281 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.123986 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr"] Sep 10 19:55:56.171396 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.171307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/342f12a4-faa4-432f-a282-19fe22752cac-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.171396 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.171364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7zcf\" (UniqueName: \"kubernetes.io/projected/342f12a4-faa4-432f-a282-19fe22752cac-kube-api-access-m7zcf\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.269668 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.269561 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-5w6xl" event={"ID":"6f7f7aff-7891-47be-b85a-0c30ead85c10","Type":"ContainerStarted","Data":"28fbe7ebdfe799a6d92a0af09aebf5c9f77bbe02365628a2e2117ca0c09ad4f1"} Sep 10 19:55:56.270298 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.270272 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:56.272750 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.271921 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"941a86b8dbeeca21339111c822aa0fbde814199e01a10d956a9c14fbc8206b09"} Sep 10 19:55:56.272750 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.272386 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/342f12a4-faa4-432f-a282-19fe22752cac-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.272750 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.272411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7zcf\" (UniqueName: \"kubernetes.io/projected/342f12a4-faa4-432f-a282-19fe22752cac-kube-api-access-m7zcf\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.272978 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:56.272777 2570 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 19:55:56.272978 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:55:56.272855 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/342f12a4-faa4-432f-a282-19fe22752cac-monitoring-plugin-cert podName:342f12a4-faa4-432f-a282-19fe22752cac nodeName:}" failed. No retries permitted until 2026-09-10 19:55:56.772824939 +0000 UTC m=+125.493772051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/342f12a4-faa4-432f-a282-19fe22752cac-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-mlxnr" (UID: "342f12a4-faa4-432f-a282-19fe22752cac") : secret "monitoring-plugin-cert" not found Sep 10 19:55:56.274336 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.274308 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xgb9" event={"ID":"1c955aee-1ab4-474e-86c3-bd3f46776c8e","Type":"ContainerStarted","Data":"0696f93a1e95b023173cc4aab9fefebff85c842ffacd7fc9956a15de36182a0f"} Sep 10 19:55:56.275923 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.275902 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" event={"ID":"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28","Type":"ContainerStarted","Data":"6bf5a665a4986fe987609d9ae59f8a75f2aa36f047078d154789d1492e943cf0"} Sep 10 19:55:56.278188 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.278159 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" event={"ID":"5d701640-ef43-4707-8982-eb3aceaba4b6","Type":"ContainerStarted","Data":"f811d805fe5939844591ee3ae47d6386873cbac76c4d0645b7d09e88dc6b2641"} Sep 10 19:55:56.278286 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.278208 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" event={"ID":"5d701640-ef43-4707-8982-eb3aceaba4b6","Type":"ContainerStarted","Data":"438c8221437c9febe105ed9b593b01520d0f818797dacd537782fac800608b9c"} Sep 10 19:55:56.278286 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.278231 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" event={"ID":"5d701640-ef43-4707-8982-eb3aceaba4b6","Type":"ContainerStarted","Data":"b345927a39e96fa9f9394bf95910bd0c6a1647ab9e1cdf1b55166488dafe740b"} Sep 10 19:55:56.283032 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.282981 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7zcf\" (UniqueName: \"kubernetes.io/projected/342f12a4-faa4-432f-a282-19fe22752cac-kube-api-access-m7zcf\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.283940 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.283896 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-5w6xl" Sep 10 19:55:56.288097 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.287782 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-5w6xl" podStartSLOduration=1.911212081 podStartE2EDuration="19.287764044s" totalImagesPullingTime="17.376551963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:56.286176903 +0000 UTC m=+125.007124024" watchObservedRunningTime="2026-09-10 19:55:56.287764044 +0000 UTC m=+125.008711167" Sep 10 19:55:56.780181 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.779548 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/342f12a4-faa4-432f-a282-19fe22752cac-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:56.784472 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:56.784415 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/342f12a4-faa4-432f-a282-19fe22752cac-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-mlxnr\" (UID: \"342f12a4-faa4-432f-a282-19fe22752cac\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:57.029154 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.029114 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:55:57.345184 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.345087 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr"] Sep 10 19:55:57.579614 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.579576 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 19:55:57.608571 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.608497 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 19:55:57.608729 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.608673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.615234 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.615207 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 19:55:57.616235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.615589 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 19:55:57.616235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.615805 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 19:55:57.616555 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.616529 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 19:55:57.617543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.616782 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 19:55:57.617543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.617020 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 19:55:57.617543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.617226 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rrs8l\"" Sep 10 19:55:57.617543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.617404 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 19:55:57.629261 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.629232 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 19:55:57.629457 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.629441 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.630213 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.630273 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.630348 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.630464 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.630539 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.630914 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5em4blqqds5pe\"" Sep 10 19:55:57.632943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.632459 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 19:55:57.688521 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688484 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688521 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688571 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688625 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688652 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-web-config\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688679 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58deb00d-8ebe-4872-8713-425e6aa0f10e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.688771 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688787 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zv2m\" (UniqueName: \"kubernetes.io/projected/58deb00d-8ebe-4872-8713-425e6aa0f10e-kube-api-access-2zv2m\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688878 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688935 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-config\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688956 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58deb00d-8ebe-4872-8713-425e6aa0f10e-config-out\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.688984 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.689016 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689112 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.689054 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689460 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.689131 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689460 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.689159 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689460 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.689189 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.689460 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.689217 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.790789 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.790741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58deb00d-8ebe-4872-8713-425e6aa0f10e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.790789 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.790794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791064 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zv2m\" (UniqueName: \"kubernetes.io/projected/58deb00d-8ebe-4872-8713-425e6aa0f10e-kube-api-access-2zv2m\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791125 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791169 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-config\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791192 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58deb00d-8ebe-4872-8713-425e6aa0f10e-config-out\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791220 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791251 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791398 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791447 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791529 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.791760 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791582 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.792705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791624 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-web-config\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.792705 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.791636 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.794988 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.794960 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.795239 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.795216 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58deb00d-8ebe-4872-8713-425e6aa0f10e-config-out\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.796153 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.795805 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.796665 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.796435 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.796665 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.796590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.797333 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.797101 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.798409 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.798090 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.802320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.800413 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.802320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.800788 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zv2m\" (UniqueName: \"kubernetes.io/projected/58deb00d-8ebe-4872-8713-425e6aa0f10e-kube-api-access-2zv2m\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.802320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.801463 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.802320 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.801963 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/58deb00d-8ebe-4872-8713-425e6aa0f10e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.803630 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.803002 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.803630 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.803581 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.804226 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.804186 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.805297 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.805119 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-web-config\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.806608 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.806556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58deb00d-8ebe-4872-8713-425e6aa0f10e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.807241 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.807198 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-config\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.808035 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.807995 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.808712 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.808688 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/58deb00d-8ebe-4872-8713-425e6aa0f10e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"58deb00d-8ebe-4872-8713-425e6aa0f10e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:57.938125 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:57.938027 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:55:58.054835 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:58.054796 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342f12a4_faa4_432f_a282_19fe22752cac.slice/crio-234e47b702eeaf1cb322419f5290c49d30ae8970a8653abd316c97e9bfe3b32e WatchSource:0}: Error finding container 234e47b702eeaf1cb322419f5290c49d30ae8970a8653abd316c97e9bfe3b32e: Status 404 returned error can't find the container with id 234e47b702eeaf1cb322419f5290c49d30ae8970a8653abd316c97e9bfe3b32e Sep 10 19:55:58.291332 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:58.291243 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" event={"ID":"342f12a4-faa4-432f-a282-19fe22752cac","Type":"ContainerStarted","Data":"234e47b702eeaf1cb322419f5290c49d30ae8970a8653abd316c97e9bfe3b32e"} Sep 10 19:55:59.504837 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:59.504714 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 19:55:59.526647 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:55:59.526558 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58deb00d_8ebe_4872_8713_425e6aa0f10e.slice/crio-c2d526785b20c09ee90b5ff2a4b43b4c74b45008ecf29ad66f75f9b7f80097a2 WatchSource:0}: Error finding container c2d526785b20c09ee90b5ff2a4b43b4c74b45008ecf29ad66f75f9b7f80097a2: Status 404 returned error can't find the container with id c2d526785b20c09ee90b5ff2a4b43b4c74b45008ecf29ad66f75f9b7f80097a2 Sep 10 19:55:59.915315 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:55:59.915259 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6ff7548c8d-d2kxn"] Sep 10 19:56:00.307525 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.307486 2570 generic.go:356] "Generic (PLEG): container finished" podID="e5fd52e1-96bc-4169-84c5-a61d15d40e04" containerID="0ac059c2cf5f0c7ff88079318fb4cf510eeb0d1267e58614a174d6c0bcbee3d3" exitCode=0 Sep 10 19:56:00.307716 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.307579 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerDied","Data":"0ac059c2cf5f0c7ff88079318fb4cf510eeb0d1267e58614a174d6c0bcbee3d3"} Sep 10 19:56:00.309856 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.309829 2570 generic.go:356] "Generic (PLEG): container finished" podID="1c955aee-1ab4-474e-86c3-bd3f46776c8e" containerID="88a538a8938c06414790c2ad134264e3ba1323982ecfd8af5715cf894b4ef407" exitCode=0 Sep 10 19:56:00.309982 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.309958 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xgb9" event={"ID":"1c955aee-1ab4-474e-86c3-bd3f46776c8e","Type":"ContainerDied","Data":"88a538a8938c06414790c2ad134264e3ba1323982ecfd8af5715cf894b4ef407"} Sep 10 19:56:00.312980 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.312958 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" event={"ID":"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28","Type":"ContainerStarted","Data":"cdaced9bb16a719b5662450bda645e6f46de1231b250a0c4ced32cf65a9b7ca5"} Sep 10 19:56:00.313254 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.313013 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" event={"ID":"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28","Type":"ContainerStarted","Data":"49a2146bb61aaed9b7cb2a0d71104023ecde3cb6179da38c015f3dc550939cb6"} Sep 10 19:56:00.313254 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.313043 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" event={"ID":"107b8bf9-9f05-469b-ac9c-9dc8e7b24f28","Type":"ContainerStarted","Data":"c96cc8d7686179d63e0254c74ddf3b7c803ff879cbe28e5ab716da960c80aed2"} Sep 10 19:56:00.315561 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.315539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" event={"ID":"5d701640-ef43-4707-8982-eb3aceaba4b6","Type":"ContainerStarted","Data":"8c2af6a3397907e21a09421d31c9fe6465316d7324174f7d90211063c79d6d7d"} Sep 10 19:56:00.317443 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.317424 2570 generic.go:356] "Generic (PLEG): container finished" podID="58deb00d-8ebe-4872-8713-425e6aa0f10e" containerID="041a6f813be4da6b441e32aea4ffb3959302f516df0acb7b4529acd9cbb3878c" exitCode=0 Sep 10 19:56:00.317535 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.317456 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerDied","Data":"041a6f813be4da6b441e32aea4ffb3959302f516df0acb7b4529acd9cbb3878c"} Sep 10 19:56:00.317535 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.317514 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"c2d526785b20c09ee90b5ff2a4b43b4c74b45008ecf29ad66f75f9b7f80097a2"} Sep 10 19:56:00.358234 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.358174 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tmsrh" podStartSLOduration=6.219285732 podStartE2EDuration="9.35814988s" totalImagesPullingTime="3.138864148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:00.356882748 +0000 UTC m=+129.077829869" watchObservedRunningTime="2026-09-10 19:56:00.35814988 +0000 UTC m=+129.079097001" Sep 10 19:56:00.381128 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:00.381055 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d248x" podStartSLOduration=6.085454111 podStartE2EDuration="9.381037084s" totalImagesPullingTime="3.295582973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:00.379316153 +0000 UTC m=+129.100263275" watchObservedRunningTime="2026-09-10 19:56:00.381037084 +0000 UTC m=+129.101984203" Sep 10 19:56:01.327691 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.327635 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xgb9" event={"ID":"1c955aee-1ab4-474e-86c3-bd3f46776c8e","Type":"ContainerStarted","Data":"099e0107b00d785c1612b8d34fa0e3da877a0c43de45f0c1cb474d6bbebc3be7"} Sep 10 19:56:01.328194 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.327710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xgb9" event={"ID":"1c955aee-1ab4-474e-86c3-bd3f46776c8e","Type":"ContainerStarted","Data":"2fa631cac80f07df779e016098cb09bebe23b3de28681af70078456be457ae50"} Sep 10 19:56:01.332865 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.332768 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" event={"ID":"342f12a4-faa4-432f-a282-19fe22752cac","Type":"ContainerStarted","Data":"c61b0b5dda1bb6b1f8814ff1fb20643c58e41b93b650ef54fe5c3ce4cfdbfe11"} Sep 10 19:56:01.332865 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.332832 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:56:01.339814 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.339786 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" Sep 10 19:56:01.353766 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.353703 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7xgb9" podStartSLOduration=6.666251544 podStartE2EDuration="10.353683741s" totalImagesPullingTime="3.571272416s" totalInitContainerRuntime="116.159781ms" isStatefulPod=true podCreationTimestamp="2026-09-10 19:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:01.34973641 +0000 UTC m=+130.070683530" watchObservedRunningTime="2026-09-10 19:56:01.353683741 +0000 UTC m=+130.074630861" Sep 10 19:56:01.368581 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.368345 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-mlxnr" podStartSLOduration=2.5986416979999998 podStartE2EDuration="5.368321266s" totalImagesPullingTime="2.769679568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:01.365900304 +0000 UTC m=+130.086847425" watchObservedRunningTime="2026-09-10 19:56:01.368321266 +0000 UTC m=+130.089268386" Sep 10 19:56:01.632459 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.632365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:56:01.635549 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.635513 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921cab07-9428-4d14-82ad-8d7cbbaa339a-metrics-certs\") pod \"network-metrics-daemon-dhzfs\" (UID: \"921cab07-9428-4d14-82ad-8d7cbbaa339a\") " pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:56:01.789242 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.788669 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mr4n2\"" Sep 10 19:56:01.793838 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:01.793815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzfs" Sep 10 19:56:02.605759 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:02.605632 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dhzfs"] Sep 10 19:56:03.352408 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:03.352328 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dhzfs" event={"ID":"921cab07-9428-4d14-82ad-8d7cbbaa339a","Type":"ContainerStarted","Data":"0971fa71f96d36d587cdefae7ca5a9c82f6a9ec3867ed09baf108de2a1175bbe"} Sep 10 19:56:03.357983 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:03.357947 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"bb101db91f7051bc90e9feafe6aed47938ccd351ea3de5f3b39d9f97be6856ea"} Sep 10 19:56:03.358159 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:03.358027 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"dba055d25c42ed4f8df7557df90b9413efec308e86e8dcfdfb33cbc51c9d8b67"} Sep 10 19:56:03.358159 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:03.358065 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"645b6553b3888498acfc831c0a466a1a39cf8c7a12bd631dd9151afe81243d08"} Sep 10 19:56:03.358159 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:03.358119 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"5c40e53166783b9fbeb0a3609db1c6835894df699e4e942ea200412a55fefffb"} Sep 10 19:56:06.372722 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:06.372686 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"386ee1aad5d208a0c8d0479087c150c0678c515e1072631824ffb09c5c262c58"} Sep 10 19:56:06.373225 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:06.372752 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"8d0085fb72e7b77ff9651aa6bc06f6e2fcc8be0481c0849cbe3e87115b627581"} Sep 10 19:56:06.375560 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:06.375530 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"a013a9b6fc4c633abffdcae511fd0edb842772e7c25f6b75316c02d110a71fd9"} Sep 10 19:56:06.377198 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:06.377174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dhzfs" event={"ID":"921cab07-9428-4d14-82ad-8d7cbbaa339a","Type":"ContainerStarted","Data":"453781a8ba1194662a16a084791f90b59d9cbb58aa27a9150f636c5e38e11370"} Sep 10 19:56:06.377299 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:06.377216 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dhzfs" event={"ID":"921cab07-9428-4d14-82ad-8d7cbbaa339a","Type":"ContainerStarted","Data":"fa3e09f25256eb6ff879be2f499bead5c6e3770cf7e8baf165d4dc0856cdd29f"} Sep 10 19:56:06.395412 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:06.395369 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dhzfs" podStartSLOduration=131.809584665 podStartE2EDuration="2m14.395355305s" totalImagesPullingTime="2.58577064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:06.3935681 +0000 UTC m=+135.114515220" watchObservedRunningTime="2026-09-10 19:56:06.395355305 +0000 UTC m=+135.116302423" Sep 10 19:56:07.386046 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:07.386007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5fd52e1-96bc-4169-84c5-a61d15d40e04","Type":"ContainerStarted","Data":"90a7641363521938718348647e731e0b1eca8cd6855d1a22f3e1a4ea0b3c65a6"} Sep 10 19:56:07.416996 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:07.416933 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.530580540999999 podStartE2EDuration="15.416911814s" totalImagesPullingTime="6.862965335s" totalInitContainerRuntime="23.365938ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:07.414858426 +0000 UTC m=+136.135805575" watchObservedRunningTime="2026-09-10 19:56:07.416911814 +0000 UTC m=+136.137858934" Sep 10 19:56:09.398259 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:09.398214 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"e0fb95548b973fbd8704fa4a7556aa65f87eb27eb0dd3138e7ed6c7ad919bc05"} Sep 10 19:56:09.398702 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:09.398343 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"45ed0e7d666e5d4db587cea63f9bc5bedd2dc918546d20c9f961001f8e1db0ab"} Sep 10 19:56:09.398702 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:09.398381 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"c34466f7397bc188407347168444229990bf3c2d15d8093675d8ca231d3170a7"} Sep 10 19:56:09.398702 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:09.398415 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"58deb00d-8ebe-4872-8713-425e6aa0f10e","Type":"ContainerStarted","Data":"7c30f0fec48e67e5606049c43781792e6109706e4e6bb9b305b4014f0e06ae4c"} Sep 10 19:56:09.430989 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:09.430928 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.660213488 podStartE2EDuration="12.430905415s" totalImagesPullingTime="7.74677118s" totalInitContainerRuntime="23.920747ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:09.428268389 +0000 UTC m=+138.149215510" watchObservedRunningTime="2026-09-10 19:56:09.430905415 +0000 UTC m=+138.151852536" Sep 10 19:56:12.460299 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:12.460265 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7456bc86c6-g455v"] Sep 10 19:56:12.938842 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:12.938808 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:56:24.942479 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:24.942415 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" podUID="8f26794b-b02c-4870-9cd2-0be56dfb642a" containerName="registry" containerID="cri-o://a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e" gracePeriod=30 Sep 10 19:56:25.222302 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.222243 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:56:25.354017 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.353986 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-image-registry-private-configuration\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354050 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-certificates\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354119 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l858w\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-kube-api-access-l858w\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354148 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f26794b-b02c-4870-9cd2-0be56dfb642a-ca-trust-extracted\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354168 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354196 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-bound-sa-token\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354235 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354225 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-installation-pull-secrets\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354538 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354308 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-trusted-ca\") pod \"8f26794b-b02c-4870-9cd2-0be56dfb642a\" (UID: \"8f26794b-b02c-4870-9cd2-0be56dfb642a\") " Sep 10 19:56:25.354870 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354841 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 19:56:25.354993 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.354967 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 19:56:25.357001 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.356969 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 19:56:25.357001 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.356992 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 19:56:25.357203 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.357123 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-kube-api-access-l858w" (OuterVolumeSpecName: "kube-api-access-l858w") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "kube-api-access-l858w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 19:56:25.357267 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.357236 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 19:56:25.357422 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.357402 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 19:56:25.362868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.362845 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f26794b-b02c-4870-9cd2-0be56dfb642a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8f26794b-b02c-4870-9cd2-0be56dfb642a" (UID: "8f26794b-b02c-4870-9cd2-0be56dfb642a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 19:56:25.455449 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455403 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-trusted-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455449 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455444 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-image-registry-private-configuration\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455449 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455455 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-certificates\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455449 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455465 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l858w\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-kube-api-access-l858w\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455707 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455474 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f26794b-b02c-4870-9cd2-0be56dfb642a-ca-trust-extracted\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455707 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455482 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-registry-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455707 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455490 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f26794b-b02c-4870-9cd2-0be56dfb642a-bound-sa-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.455707 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.455497 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f26794b-b02c-4870-9cd2-0be56dfb642a-installation-pull-secrets\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:25.468864 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.468836 2570 generic.go:356] "Generic (PLEG): container finished" podID="8f26794b-b02c-4870-9cd2-0be56dfb642a" containerID="a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e" exitCode=0 Sep 10 19:56:25.468993 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.468889 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" Sep 10 19:56:25.468993 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.468903 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" event={"ID":"8f26794b-b02c-4870-9cd2-0be56dfb642a","Type":"ContainerDied","Data":"a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e"} Sep 10 19:56:25.468993 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.468942 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ff7548c8d-d2kxn" event={"ID":"8f26794b-b02c-4870-9cd2-0be56dfb642a","Type":"ContainerDied","Data":"e55fc6b4a0c1ba8efa4b17840e803334f5f59ac0324557eb303d7ded4a5883c4"} Sep 10 19:56:25.468993 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.468962 2570 scope.go:117] "RemoveContainer" containerID="a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e" Sep 10 19:56:25.477790 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.477774 2570 scope.go:117] "RemoveContainer" containerID="a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e" Sep 10 19:56:25.478057 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:56:25.478037 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e\": container with ID starting with a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e not found: ID does not exist" containerID="a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e" Sep 10 19:56:25.478172 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.478065 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e"} err="failed to get container status \"a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e\": rpc error: code = NotFound desc = could not find container \"a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e\": container with ID starting with a00c4a1cc4a413399ebe9396e7a425b18ce1fd9930cd2b426dcb15a1bafbd75e not found: ID does not exist" Sep 10 19:56:25.490994 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.490968 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6ff7548c8d-d2kxn"] Sep 10 19:56:25.493324 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.493304 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6ff7548c8d-d2kxn"] Sep 10 19:56:25.764542 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:25.764512 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f26794b-b02c-4870-9cd2-0be56dfb642a" path="/var/lib/kubelet/pods/8f26794b-b02c-4870-9cd2-0be56dfb642a/volumes" Sep 10 19:56:26.474921 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:26.474826 2570 generic.go:356] "Generic (PLEG): container finished" podID="d4d3c460-eb05-4132-b042-b961335105d9" containerID="8f6862929f15009940b537e6aaf664338a09681d553f7ace7daceb831d60a93b" exitCode=0 Sep 10 19:56:26.474921 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:26.474890 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" event={"ID":"d4d3c460-eb05-4132-b042-b961335105d9","Type":"ContainerDied","Data":"8f6862929f15009940b537e6aaf664338a09681d553f7ace7daceb831d60a93b"} Sep 10 19:56:26.475442 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:26.475293 2570 scope.go:117] "RemoveContainer" containerID="8f6862929f15009940b537e6aaf664338a09681d553f7ace7daceb831d60a93b" Sep 10 19:56:27.481303 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:27.481264 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pf2cw" event={"ID":"d4d3c460-eb05-4132-b042-b961335105d9","Type":"ContainerStarted","Data":"7a78e3af2a0fff1eca16a2bb75ecd3b85b3d41c8f9bad92fb143a563ffbaefe7"} Sep 10 19:56:37.487198 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.487160 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7456bc86c6-g455v" podUID="4a4d39c9-8a77-415f-85f8-2af94596c0ed" containerName="console" containerID="cri-o://534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4" gracePeriod=15 Sep 10 19:56:37.769369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.769342 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:56:37.859509 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859477 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-oauth-config\") pod \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " Sep 10 19:56:37.859509 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859511 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgmf6\" (UniqueName: \"kubernetes.io/projected/4a4d39c9-8a77-415f-85f8-2af94596c0ed-kube-api-access-jgmf6\") pod \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " Sep 10 19:56:37.859735 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859527 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-config\") pod \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " Sep 10 19:56:37.859735 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859549 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-serving-cert\") pod \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " Sep 10 19:56:37.859735 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859583 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-oauth-serving-cert\") pod \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " Sep 10 19:56:37.859735 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859722 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-service-ca\") pod \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\" (UID: \"4a4d39c9-8a77-415f-85f8-2af94596c0ed\") " Sep 10 19:56:37.860015 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859980 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-config" (OuterVolumeSpecName: "console-config") pod "4a4d39c9-8a77-415f-85f8-2af94596c0ed" (UID: "4a4d39c9-8a77-415f-85f8-2af94596c0ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 19:56:37.860015 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.859989 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a4d39c9-8a77-415f-85f8-2af94596c0ed" (UID: "4a4d39c9-8a77-415f-85f8-2af94596c0ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 19:56:37.860214 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.860057 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a4d39c9-8a77-415f-85f8-2af94596c0ed" (UID: "4a4d39c9-8a77-415f-85f8-2af94596c0ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 19:56:37.861956 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.861929 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4d39c9-8a77-415f-85f8-2af94596c0ed-kube-api-access-jgmf6" (OuterVolumeSpecName: "kube-api-access-jgmf6") pod "4a4d39c9-8a77-415f-85f8-2af94596c0ed" (UID: "4a4d39c9-8a77-415f-85f8-2af94596c0ed"). InnerVolumeSpecName "kube-api-access-jgmf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 19:56:37.862057 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.861953 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a4d39c9-8a77-415f-85f8-2af94596c0ed" (UID: "4a4d39c9-8a77-415f-85f8-2af94596c0ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 19:56:37.862057 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.861998 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a4d39c9-8a77-415f-85f8-2af94596c0ed" (UID: "4a4d39c9-8a77-415f-85f8-2af94596c0ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 19:56:37.961087 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.961046 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:37.961263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.961100 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-oauth-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:37.961263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.961111 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgmf6\" (UniqueName: \"kubernetes.io/projected/4a4d39c9-8a77-415f-85f8-2af94596c0ed-kube-api-access-jgmf6\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:37.961263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.961120 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:37.961263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.961129 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a4d39c9-8a77-415f-85f8-2af94596c0ed-console-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:37.961263 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:37.961136 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a4d39c9-8a77-415f-85f8-2af94596c0ed-oauth-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 19:56:38.532113 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.532064 2570 generic.go:356] "Generic (PLEG): container finished" podID="4a4d39c9-8a77-415f-85f8-2af94596c0ed" containerID="534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4" exitCode=0 Sep 10 19:56:38.532113 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.532103 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456bc86c6-g455v" event={"ID":"4a4d39c9-8a77-415f-85f8-2af94596c0ed","Type":"ContainerDied","Data":"534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4"} Sep 10 19:56:38.532624 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.532138 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7456bc86c6-g455v" Sep 10 19:56:38.532624 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.532165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7456bc86c6-g455v" event={"ID":"4a4d39c9-8a77-415f-85f8-2af94596c0ed","Type":"ContainerDied","Data":"c096d546275b11f1e73e1f8d2e0373706105a368474009530ed35be0c2bca83a"} Sep 10 19:56:38.532624 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.532193 2570 scope.go:117] "RemoveContainer" containerID="534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4" Sep 10 19:56:38.543927 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.543911 2570 scope.go:117] "RemoveContainer" containerID="534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4" Sep 10 19:56:38.544218 ip-10-0-130-59 kubenswrapper[2570]: E0910 19:56:38.544195 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4\": container with ID starting with 534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4 not found: ID does not exist" containerID="534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4" Sep 10 19:56:38.544288 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.544228 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4"} err="failed to get container status \"534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4\": rpc error: code = NotFound desc = could not find container \"534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4\": container with ID starting with 534525c4db00d6159f65614d673ab8499373c273cd5a8856a1b5b70c0caf69a4 not found: ID does not exist" Sep 10 19:56:38.553290 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.553268 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7456bc86c6-g455v"] Sep 10 19:56:38.557449 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:38.557431 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7456bc86c6-g455v"] Sep 10 19:56:39.765597 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:39.765562 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a4d39c9-8a77-415f-85f8-2af94596c0ed" path="/var/lib/kubelet/pods/4a4d39c9-8a77-415f-85f8-2af94596c0ed/volumes" Sep 10 19:56:57.939018 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:57.938983 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:56:57.958091 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:57.958044 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:56:58.661365 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:56:58.661336 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 19:57:15.786548 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786515 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc"] Sep 10 19:57:15.787026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786864 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4d39c9-8a77-415f-85f8-2af94596c0ed" containerName="console" Sep 10 19:57:15.787026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786875 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4d39c9-8a77-415f-85f8-2af94596c0ed" containerName="console" Sep 10 19:57:15.787026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786889 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f26794b-b02c-4870-9cd2-0be56dfb642a" containerName="registry" Sep 10 19:57:15.787026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786896 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f26794b-b02c-4870-9cd2-0be56dfb642a" containerName="registry" Sep 10 19:57:15.787026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786943 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4d39c9-8a77-415f-85f8-2af94596c0ed" containerName="console" Sep 10 19:57:15.787026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.786953 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f26794b-b02c-4870-9cd2-0be56dfb642a" containerName="registry" Sep 10 19:57:15.789259 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.789238 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.792266 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.792240 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 19:57:15.792394 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.792304 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 19:57:15.792394 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.792249 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fvc44\"" Sep 10 19:57:15.792394 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.792359 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 19:57:15.792648 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.792633 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 19:57:15.794848 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.794828 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 19:57:15.798552 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.798531 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 19:57:15.810173 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.810150 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc"] Sep 10 19:57:15.873044 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873003 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-serving-certs-ca-bundle\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873044 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873042 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-secret-telemeter-client\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873278 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873089 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873278 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873193 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873278 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873232 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-telemeter-client-tls\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873431 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873336 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grl8q\" (UniqueName: \"kubernetes.io/projected/dddec83d-7d0e-4692-9338-25b97ef8b445-kube-api-access-grl8q\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873431 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873378 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-metrics-client-ca\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.873431 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.873417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-federate-client-tls\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974386 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-telemeter-client-tls\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grl8q\" (UniqueName: \"kubernetes.io/projected/dddec83d-7d0e-4692-9338-25b97ef8b445-kube-api-access-grl8q\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-metrics-client-ca\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974461 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-federate-client-tls\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-serving-certs-ca-bundle\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-secret-telemeter-client\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974512 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.974586 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.974550 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.975379 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.975345 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-metrics-client-ca\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.975501 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.975345 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-serving-certs-ca-bundle\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.975569 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.975547 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dddec83d-7d0e-4692-9338-25b97ef8b445-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.977219 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.977194 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-federate-client-tls\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.977317 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.977194 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-telemeter-client-tls\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.977523 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.977500 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.977559 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.977517 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dddec83d-7d0e-4692-9338-25b97ef8b445-secret-telemeter-client\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:15.990652 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:15.990629 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grl8q\" (UniqueName: \"kubernetes.io/projected/dddec83d-7d0e-4692-9338-25b97ef8b445-kube-api-access-grl8q\") pod \"telemeter-client-6c94f4cbbf-2mlsc\" (UID: \"dddec83d-7d0e-4692-9338-25b97ef8b445\") " pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:16.099427 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:16.099339 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" Sep 10 19:57:16.376166 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:16.376134 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc"] Sep 10 19:57:16.378369 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:57:16.378342 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddec83d_7d0e_4692_9338_25b97ef8b445.slice/crio-29c8dfa7ed8c90344ea2a8ef1675f32b7a2f7ca76a8a8095e748950a19ad14a1 WatchSource:0}: Error finding container 29c8dfa7ed8c90344ea2a8ef1675f32b7a2f7ca76a8a8095e748950a19ad14a1: Status 404 returned error can't find the container with id 29c8dfa7ed8c90344ea2a8ef1675f32b7a2f7ca76a8a8095e748950a19ad14a1 Sep 10 19:57:16.716027 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:16.715934 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" event={"ID":"dddec83d-7d0e-4692-9338-25b97ef8b445","Type":"ContainerStarted","Data":"29c8dfa7ed8c90344ea2a8ef1675f32b7a2f7ca76a8a8095e748950a19ad14a1"} Sep 10 19:57:18.726685 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:18.726647 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" event={"ID":"dddec83d-7d0e-4692-9338-25b97ef8b445","Type":"ContainerStarted","Data":"2095487d839e7d16fc1f7d1fffff3d06b90f938ee95e40449c850e5e6c83bd2b"} Sep 10 19:57:18.727098 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:18.726712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" event={"ID":"dddec83d-7d0e-4692-9338-25b97ef8b445","Type":"ContainerStarted","Data":"e60c86cf987ca7bbc2c1856dfa91f4c2dc5e57f056b5d3df0791854e453618da"} Sep 10 19:57:18.727098 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:18.726743 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" event={"ID":"dddec83d-7d0e-4692-9338-25b97ef8b445","Type":"ContainerStarted","Data":"4665505f6b83e4fe78ef7f8e6c1f5621bc020bb97cb2834d868bbed09349595b"} Sep 10 19:57:18.755354 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:57:18.755306 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c94f4cbbf-2mlsc" podStartSLOduration=2.134476594 podStartE2EDuration="3.755290543s" totalImagesPullingTime="1.620813949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:57:18.754493342 +0000 UTC m=+207.475440461" watchObservedRunningTime="2026-09-10 19:57:18.755290543 +0000 UTC m=+207.476237756" Sep 10 19:58:51.720927 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:58:51.720901 2570 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 19:59:17.147688 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.147651 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm"] Sep 10 19:59:17.155521 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.155492 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.159900 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.159878 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 19:59:17.160402 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.160385 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gxjbr\"" Sep 10 19:59:17.160682 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.160665 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 19:59:17.163119 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.162922 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 19:59:17.166432 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.166414 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 19:59:17.190217 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.190192 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm"] Sep 10 19:59:17.195957 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.195928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a0eb861-1be8-455d-ae15-092de7e963d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.196096 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.196003 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a0eb861-1be8-455d-ae15-092de7e963d7-manifests\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.196096 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.196050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0eb861-1be8-455d-ae15-092de7e963d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.196183 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.196122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a0eb861-1be8-455d-ae15-092de7e963d7-tmp\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.196183 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.196150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkrf9\" (UniqueName: \"kubernetes.io/projected/1a0eb861-1be8-455d-ae15-092de7e963d7-kube-api-access-bkrf9\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.297345 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.297315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a0eb861-1be8-455d-ae15-092de7e963d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.297531 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.297365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a0eb861-1be8-455d-ae15-092de7e963d7-manifests\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.297531 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.297423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0eb861-1be8-455d-ae15-092de7e963d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.297531 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.297469 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a0eb861-1be8-455d-ae15-092de7e963d7-tmp\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.297531 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.297493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkrf9\" (UniqueName: \"kubernetes.io/projected/1a0eb861-1be8-455d-ae15-092de7e963d7-kube-api-access-bkrf9\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.297908 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.297877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a0eb861-1be8-455d-ae15-092de7e963d7-manifests\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.299718 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.299687 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a0eb861-1be8-455d-ae15-092de7e963d7-tmp\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.299945 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.299927 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0eb861-1be8-455d-ae15-092de7e963d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.300021 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.300003 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a0eb861-1be8-455d-ae15-092de7e963d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.314221 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.314197 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkrf9\" (UniqueName: \"kubernetes.io/projected/1a0eb861-1be8-455d-ae15-092de7e963d7-kube-api-access-bkrf9\") pod \"opendatahub-operator-controller-manager-5678c4c4c5-s6zwm\" (UID: \"1a0eb861-1be8-455d-ae15-092de7e963d7\") " pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.466943 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.466857 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:17.776496 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.776466 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm"] Sep 10 19:59:17.778671 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:59:17.778642 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0eb861_1be8_455d_ae15_092de7e963d7.slice/crio-f8753cc86c7b311f751f6ea350a24d8ef1f955b5d58ec3392b91117340ec07f2 WatchSource:0}: Error finding container f8753cc86c7b311f751f6ea350a24d8ef1f955b5d58ec3392b91117340ec07f2: Status 404 returned error can't find the container with id f8753cc86c7b311f751f6ea350a24d8ef1f955b5d58ec3392b91117340ec07f2 Sep 10 19:59:17.780375 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:17.780359 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 19:59:18.209633 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:18.209546 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" event={"ID":"1a0eb861-1be8-455d-ae15-092de7e963d7","Type":"ContainerStarted","Data":"f8753cc86c7b311f751f6ea350a24d8ef1f955b5d58ec3392b91117340ec07f2"} Sep 10 19:59:21.225978 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:21.225938 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" event={"ID":"1a0eb861-1be8-455d-ae15-092de7e963d7","Type":"ContainerStarted","Data":"72bdfbf3f7db19d6a766766022d8987de17cd27ee9666d5432db24014fb59025"} Sep 10 19:59:22.237949 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:22.237917 2570 generic.go:356] "Generic (PLEG): container finished" podID="1a0eb861-1be8-455d-ae15-092de7e963d7" containerID="72bdfbf3f7db19d6a766766022d8987de17cd27ee9666d5432db24014fb59025" exitCode=0 Sep 10 19:59:22.238378 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:22.237997 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" event={"ID":"1a0eb861-1be8-455d-ae15-092de7e963d7","Type":"ContainerDied","Data":"72bdfbf3f7db19d6a766766022d8987de17cd27ee9666d5432db24014fb59025"} Sep 10 19:59:23.244455 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:23.244424 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" event={"ID":"1a0eb861-1be8-455d-ae15-092de7e963d7","Type":"ContainerStarted","Data":"12f28decf1d64591ceaa2e7288f28cda632d53718c3bf07100f4eff24c9c8b25"} Sep 10 19:59:23.244913 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:23.244547 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:23.269758 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:23.269708 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" podStartSLOduration=2.006238079 podStartE2EDuration="6.269692377s" totalImagesPullingTime="3.173709151s" totalInitContainerRuntime="1.089745147s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:59:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:59:23.268202795 +0000 UTC m=+331.989149914" watchObservedRunningTime="2026-09-10 19:59:23.269692377 +0000 UTC m=+331.990639495" Sep 10 19:59:34.251013 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:34.250983 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5678c4c4c5-s6zwm" Sep 10 19:59:50.020462 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.020389 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt"] Sep 10 19:59:50.023868 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.023845 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.027369 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.027353 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 19:59:50.027622 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.027605 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 19:59:50.027744 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.027731 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 19:59:50.027873 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.027855 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rhdxm\"" Sep 10 19:59:50.045879 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.045859 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt"] Sep 10 19:59:50.181486 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181486 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181494 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181517 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181637 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrjhl\" (UniqueName: \"kubernetes.io/projected/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-kube-api-access-wrjhl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181706 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181747 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.181968 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.181779 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282560 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrjhl\" (UniqueName: \"kubernetes.io/projected/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-kube-api-access-wrjhl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282560 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282560 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282536 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282560 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282559 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282881 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282881 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282881 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282646 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282881 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282701 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.282881 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.282737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.283179 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.283056 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.283179 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.283155 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.283297 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.283174 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.283352 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.283339 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.283542 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.283521 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.285102 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.285061 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.285403 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.285386 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.291371 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.291348 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrjhl\" (UniqueName: \"kubernetes.io/projected/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-kube-api-access-wrjhl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.291449 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.291381 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt\" (UID: \"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.334453 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.334425 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:50.612996 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:50.612907 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt"] Sep 10 19:59:50.616313 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:59:50.616281 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab17c8e0_4c7d_4cbd_8b93_7eb356dd4385.slice/crio-9de4034ade269d3479a2abd76097ad38205abb9a8f4f464a28e851ac9d753afc WatchSource:0}: Error finding container 9de4034ade269d3479a2abd76097ad38205abb9a8f4f464a28e851ac9d753afc: Status 404 returned error can't find the container with id 9de4034ade269d3479a2abd76097ad38205abb9a8f4f464a28e851ac9d753afc Sep 10 19:59:51.366994 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:51.366949 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" event={"ID":"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385","Type":"ContainerStarted","Data":"9de4034ade269d3479a2abd76097ad38205abb9a8f4f464a28e851ac9d753afc"} Sep 10 19:59:53.170475 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:53.170429 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 19:59:53.170847 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:53.170519 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 19:59:53.170847 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:53.170564 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 19:59:53.378554 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:53.378513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" event={"ID":"ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385","Type":"ContainerStarted","Data":"6df5969c0e51f70b78d874a89cc1f3cb832a1fb73cfe43b17b78da87f3951274"} Sep 10 19:59:53.403662 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:53.403607 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" podStartSLOduration=1.8520884149999999 podStartE2EDuration="4.40359173s" totalImagesPullingTime="2.551503315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:59:53.401597452 +0000 UTC m=+362.122544572" watchObservedRunningTime="2026-09-10 19:59:53.40359173 +0000 UTC m=+362.124538849" Sep 10 19:59:54.335136 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.335102 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:54.339529 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.339508 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:54.383249 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.383222 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:54.384183 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.384164 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt" Sep 10 19:59:54.824026 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.823994 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n"] Sep 10 19:59:54.827626 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.827606 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:54.831591 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.831571 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ss24s\"" Sep 10 19:59:54.831697 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.831620 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 19:59:54.831983 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.831965 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 19:59:54.832058 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.831968 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 19:59:54.832058 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.832041 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 19:59:54.832592 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.832577 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 19:59:54.844518 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.844487 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n"] Sep 10 19:59:54.924647 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.924619 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlkxm\" (UniqueName: \"kubernetes.io/projected/22a9b59a-243d-4dc2-ad00-f73486afd675-kube-api-access-vlkxm\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:54.924789 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.924672 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/22a9b59a-243d-4dc2-ad00-f73486afd675-manager-config\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:54.924789 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.924696 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/22a9b59a-243d-4dc2-ad00-f73486afd675-metrics-cert\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:54.924789 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:54.924744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22a9b59a-243d-4dc2-ad00-f73486afd675-cert\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.025500 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.025464 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/22a9b59a-243d-4dc2-ad00-f73486afd675-manager-config\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.025500 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.025497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/22a9b59a-243d-4dc2-ad00-f73486afd675-metrics-cert\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.025718 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.025562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22a9b59a-243d-4dc2-ad00-f73486afd675-cert\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.025718 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.025615 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlkxm\" (UniqueName: \"kubernetes.io/projected/22a9b59a-243d-4dc2-ad00-f73486afd675-kube-api-access-vlkxm\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.026242 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.026220 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/22a9b59a-243d-4dc2-ad00-f73486afd675-manager-config\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.028178 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.028152 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/22a9b59a-243d-4dc2-ad00-f73486afd675-metrics-cert\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.028273 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.028187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22a9b59a-243d-4dc2-ad00-f73486afd675-cert\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.037047 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.037023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlkxm\" (UniqueName: \"kubernetes.io/projected/22a9b59a-243d-4dc2-ad00-f73486afd675-kube-api-access-vlkxm\") pod \"lws-controller-manager-6fd976b7bd-pn47n\" (UID: \"22a9b59a-243d-4dc2-ad00-f73486afd675\") " pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.063493 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.063467 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g"] Sep 10 19:59:55.066939 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.066924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.071993 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.071977 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 19:59:55.075941 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.075888 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hb8rj\"" Sep 10 19:59:55.076136 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.076115 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 19:59:55.082216 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.082202 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 19:59:55.137228 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.137200 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 19:59:55.157991 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.157961 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g"] Sep 10 19:59:55.226885 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.226854 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.226885 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.226888 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.227119 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.226920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.227119 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.227004 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plk59\" (UniqueName: \"kubernetes.io/projected/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-kube-api-access-plk59\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.227119 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.227056 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.227268 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.227121 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.227268 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.227148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.328622 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.328622 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328569 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plk59\" (UniqueName: \"kubernetes.io/projected/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-kube-api-access-plk59\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.328622 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.328929 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328624 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.328929 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328790 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.328929 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328914 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.329114 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.328942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.329413 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.329377 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.331042 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.331021 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.331185 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.331170 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.331439 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.331417 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.331582 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.331565 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.337860 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.337841 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.338526 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.338508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plk59\" (UniqueName: \"kubernetes.io/projected/2e92d525-ff7c-46d2-b1e8-861d0b37db1e-kube-api-access-plk59\") pod \"istiod-openshift-gateway-5677ccb49b-l9w4g\" (UID: \"2e92d525-ff7c-46d2-b1e8-861d0b37db1e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.377243 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.377210 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:55.420657 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.420566 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n"] Sep 10 19:59:55.422762 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:59:55.422738 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a9b59a_243d_4dc2_ad00_f73486afd675.slice/crio-e295ac7c3ecc6785024b8ae1087e80d57731efe39ec72e4edb81ee31315d31b5 WatchSource:0}: Error finding container e295ac7c3ecc6785024b8ae1087e80d57731efe39ec72e4edb81ee31315d31b5: Status 404 returned error can't find the container with id e295ac7c3ecc6785024b8ae1087e80d57731efe39ec72e4edb81ee31315d31b5 Sep 10 19:59:55.677429 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:55.677357 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g"] Sep 10 19:59:55.679507 ip-10-0-130-59 kubenswrapper[2570]: W0910 19:59:55.679483 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e92d525_ff7c_46d2_b1e8_861d0b37db1e.slice/crio-6cdb50a7e5d6c7a3c51b5dc22560fdd1d7cc2dc53b5737cef227b8c54b78298f WatchSource:0}: Error finding container 6cdb50a7e5d6c7a3c51b5dc22560fdd1d7cc2dc53b5737cef227b8c54b78298f: Status 404 returned error can't find the container with id 6cdb50a7e5d6c7a3c51b5dc22560fdd1d7cc2dc53b5737cef227b8c54b78298f Sep 10 19:59:56.394283 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:56.394244 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" event={"ID":"2e92d525-ff7c-46d2-b1e8-861d0b37db1e","Type":"ContainerStarted","Data":"6cdb50a7e5d6c7a3c51b5dc22560fdd1d7cc2dc53b5737cef227b8c54b78298f"} Sep 10 19:59:56.395785 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:56.395736 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" event={"ID":"22a9b59a-243d-4dc2-ad00-f73486afd675","Type":"ContainerStarted","Data":"e295ac7c3ecc6785024b8ae1087e80d57731efe39ec72e4edb81ee31315d31b5"} Sep 10 19:59:58.127107 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:58.127055 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 19:59:58.127386 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:58.127142 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 19:59:58.407409 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:58.407325 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" event={"ID":"2e92d525-ff7c-46d2-b1e8-861d0b37db1e","Type":"ContainerStarted","Data":"3f1130f8d270c7636a41379d6f24d30ae750b91112de55a38f713b6a285225c5"} Sep 10 19:59:58.407543 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:58.407453 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 19:59:58.476684 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:58.476633 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" podStartSLOduration=1.031403735 podStartE2EDuration="3.476616064s" totalImagesPullingTime="2.445212329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:59:58.475268531 +0000 UTC m=+367.196215649" watchObservedRunningTime="2026-09-10 19:59:58.476616064 +0000 UTC m=+367.197563182" Sep 10 19:59:59.415770 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:59.415739 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-l9w4g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 19:59:59.416180 ip-10-0-130-59 kubenswrapper[2570]: I0910 19:59:59.415794 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" podUID="2e92d525-ff7c-46d2-b1e8-861d0b37db1e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 20:00:02.414592 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:00:02.414557 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l9w4g" Sep 10 20:00:25.532131 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:00:25.532089 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" event={"ID":"22a9b59a-243d-4dc2-ad00-f73486afd675","Type":"ContainerStarted","Data":"43524dab87df6e82ad0d0ed705b5173d8392e3c82000d77de29251b0fb70b55d"} Sep 10 20:00:25.532521 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:00:25.532228 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 20:00:25.553128 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:00:25.553054 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" podStartSLOduration=1.539294634 podStartE2EDuration="31.553037592s" totalImagesPullingTime="30.013742958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:00:25.552815064 +0000 UTC m=+394.273762185" watchObservedRunningTime="2026-09-10 20:00:25.553037592 +0000 UTC m=+394.273984712" Sep 10 20:00:36.539558 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:00:36.539526 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6fd976b7bd-pn47n" Sep 10 20:01:34.003663 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.003631 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz"] Sep 10 20:01:34.006744 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.006714 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.011522 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.011499 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 20:01:34.011764 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.011687 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fnjn8\"" Sep 10 20:01:34.011867 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.011849 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 20:01:34.041465 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.041430 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz"] Sep 10 20:01:34.099794 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.099765 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a090ded7-60bb-484a-aedf-36b06ae6829f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-94hrz\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.099938 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.099819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n8s6\" (UniqueName: \"kubernetes.io/projected/a090ded7-60bb-484a-aedf-36b06ae6829f-kube-api-access-6n8s6\") pod \"kuadrant-operator-controller-manager-5769f99d9f-94hrz\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.200876 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.200840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a090ded7-60bb-484a-aedf-36b06ae6829f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-94hrz\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.201035 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.200900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n8s6\" (UniqueName: \"kubernetes.io/projected/a090ded7-60bb-484a-aedf-36b06ae6829f-kube-api-access-6n8s6\") pod \"kuadrant-operator-controller-manager-5769f99d9f-94hrz\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.201252 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.201233 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a090ded7-60bb-484a-aedf-36b06ae6829f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-94hrz\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.211219 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.211192 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n8s6\" (UniqueName: \"kubernetes.io/projected/a090ded7-60bb-484a-aedf-36b06ae6829f-kube-api-access-6n8s6\") pod \"kuadrant-operator-controller-manager-5769f99d9f-94hrz\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.318167 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.318092 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:34.597507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.597431 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz"] Sep 10 20:01:34.601999 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:01:34.601970 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda090ded7_60bb_484a_aedf_36b06ae6829f.slice/crio-36cde840aee305b0a709c493c35fa2a04e5ec148510ee617f2e222e0db6a1a2e WatchSource:0}: Error finding container 36cde840aee305b0a709c493c35fa2a04e5ec148510ee617f2e222e0db6a1a2e: Status 404 returned error can't find the container with id 36cde840aee305b0a709c493c35fa2a04e5ec148510ee617f2e222e0db6a1a2e Sep 10 20:01:34.857195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:34.857098 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" event={"ID":"a090ded7-60bb-484a-aedf-36b06ae6829f","Type":"ContainerStarted","Data":"36cde840aee305b0a709c493c35fa2a04e5ec148510ee617f2e222e0db6a1a2e"} Sep 10 20:01:38.881821 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:38.881779 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" event={"ID":"a090ded7-60bb-484a-aedf-36b06ae6829f","Type":"ContainerStarted","Data":"92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316"} Sep 10 20:01:38.882229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:38.881850 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:49.888852 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:49.888817 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:01:49.912150 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:01:49.912064 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" podStartSLOduration=13.064417656 podStartE2EDuration="16.912041782s" totalImagesPullingTime="3.847624126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:01:38.927281257 +0000 UTC m=+467.648228376" watchObservedRunningTime="2026-09-10 20:01:49.912041782 +0000 UTC m=+478.632988901" Sep 10 20:02:45.391936 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.391843 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7769777fc8-jpjrf"] Sep 10 20:02:45.394449 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.394427 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.397430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.397394 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 20:02:45.397430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.397420 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9564f\"" Sep 10 20:02:45.397952 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.397933 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 20:02:45.398045 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.398013 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 20:02:45.406912 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.406889 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7769777fc8-jpjrf"] Sep 10 20:02:45.457293 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.457256 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.457500 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.457330 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-metrics-certs\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.457500 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.457417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/20f10edc-c486-4e27-a194-00b625329529-kube-api-access-dqghz\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.558549 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.558515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/20f10edc-c486-4e27-a194-00b625329529-kube-api-access-dqghz\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.558716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.558573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.558716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.558611 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-metrics-certs\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.558791 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:02:45.558738 2570 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 20:02:45.558842 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:02:45.558830 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert podName:20f10edc-c486-4e27-a194-00b625329529 nodeName:}" failed. No retries permitted until 2026-09-10 20:02:46.058808019 +0000 UTC m=+534.779755131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert") pod "maas-controller-7769777fc8-jpjrf" (UID: "20f10edc-c486-4e27-a194-00b625329529") : secret "maas-controller-webhook-cert" not found Sep 10 20:02:45.561117 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.561059 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-metrics-certs\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.570307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.570282 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/20f10edc-c486-4e27-a194-00b625329529-kube-api-access-dqghz\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:45.580127 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.580102 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fc475458-vq44r"] Sep 10 20:02:45.582669 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.582652 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.596459 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.596435 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc475458-vq44r"] Sep 10 20:02:45.659752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.659661 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-metrics-certs\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.659752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.659710 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.659941 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.659818 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg5b6\" (UniqueName: \"kubernetes.io/projected/e8532c6d-0e25-46da-b999-cbffa8cda1bb-kube-api-access-kg5b6\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.760325 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.760286 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-metrics-certs\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.760520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.760340 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.760520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.760397 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg5b6\" (UniqueName: \"kubernetes.io/projected/e8532c6d-0e25-46da-b999-cbffa8cda1bb-kube-api-access-kg5b6\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.760637 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:02:45.760530 2570 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 20:02:45.760637 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:02:45.760630 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert podName:e8532c6d-0e25-46da-b999-cbffa8cda1bb nodeName:}" failed. No retries permitted until 2026-09-10 20:02:46.260607125 +0000 UTC m=+534.981554225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert") pod "maas-controller-7fc475458-vq44r" (UID: "e8532c6d-0e25-46da-b999-cbffa8cda1bb") : secret "maas-controller-webhook-cert" not found Sep 10 20:02:45.762954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.762928 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-metrics-certs\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:45.769355 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:45.769333 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg5b6\" (UniqueName: \"kubernetes.io/projected/e8532c6d-0e25-46da-b999-cbffa8cda1bb-kube-api-access-kg5b6\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:46.063940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:46.063900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:46.064146 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:02:46.064054 2570 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 20:02:46.064196 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:02:46.064164 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert podName:20f10edc-c486-4e27-a194-00b625329529 nodeName:}" failed. No retries permitted until 2026-09-10 20:02:47.064147719 +0000 UTC m=+535.785094816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert") pod "maas-controller-7769777fc8-jpjrf" (UID: "20f10edc-c486-4e27-a194-00b625329529") : secret "maas-controller-webhook-cert" not found Sep 10 20:02:46.266536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:46.266493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:46.268959 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:46.268928 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert\") pod \"maas-controller-7fc475458-vq44r\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:46.492990 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:46.492958 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:46.765286 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:46.765256 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc475458-vq44r"] Sep 10 20:02:46.767131 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:02:46.767103 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8532c6d_0e25_46da_b999_cbffa8cda1bb.slice/crio-3aa12dc646e9c42f1527051e5827d4129f1f175e13e79018d81c7172a3df2dac WatchSource:0}: Error finding container 3aa12dc646e9c42f1527051e5827d4129f1f175e13e79018d81c7172a3df2dac: Status 404 returned error can't find the container with id 3aa12dc646e9c42f1527051e5827d4129f1f175e13e79018d81c7172a3df2dac Sep 10 20:02:47.073124 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:47.073003 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:47.075412 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:47.075384 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") pod \"maas-controller-7769777fc8-jpjrf\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:47.202148 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:47.202104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-vq44r" event={"ID":"e8532c6d-0e25-46da-b999-cbffa8cda1bb","Type":"ContainerStarted","Data":"3aa12dc646e9c42f1527051e5827d4129f1f175e13e79018d81c7172a3df2dac"} Sep 10 20:02:47.205343 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:47.205313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:47.481309 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:47.481278 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7769777fc8-jpjrf"] Sep 10 20:02:47.483591 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:02:47.483556 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f10edc_c486_4e27_a194_00b625329529.slice/crio-cdf9cf708401baaa541a21073b5aa07ad9782c15e7206f5e02bea11f4976243b WatchSource:0}: Error finding container cdf9cf708401baaa541a21073b5aa07ad9782c15e7206f5e02bea11f4976243b: Status 404 returned error can't find the container with id cdf9cf708401baaa541a21073b5aa07ad9782c15e7206f5e02bea11f4976243b Sep 10 20:02:48.212105 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:48.212011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-jpjrf" event={"ID":"20f10edc-c486-4e27-a194-00b625329529","Type":"ContainerStarted","Data":"cdf9cf708401baaa541a21073b5aa07ad9782c15e7206f5e02bea11f4976243b"} Sep 10 20:02:50.222745 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:50.222709 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-vq44r" event={"ID":"e8532c6d-0e25-46da-b999-cbffa8cda1bb","Type":"ContainerStarted","Data":"4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa"} Sep 10 20:02:50.223234 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:50.222810 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:02:50.224334 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:50.224311 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-jpjrf" event={"ID":"20f10edc-c486-4e27-a194-00b625329529","Type":"ContainerStarted","Data":"d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef"} Sep 10 20:02:50.224443 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:50.224357 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:02:50.275215 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:50.275158 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7769777fc8-jpjrf" podStartSLOduration=3.296809479 podStartE2EDuration="5.275139149s" totalImagesPullingTime="1.97832967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:02:50.273477579 +0000 UTC m=+538.994424698" watchObservedRunningTime="2026-09-10 20:02:50.275139149 +0000 UTC m=+538.996086268" Sep 10 20:02:50.276592 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:02:50.276549 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fc475458-vq44r" podStartSLOduration=2.590461697 podStartE2EDuration="5.276538054s" totalImagesPullingTime="2.686076357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:02:50.251909854 +0000 UTC m=+538.972856985" watchObservedRunningTime="2026-09-10 20:02:50.276538054 +0000 UTC m=+538.997485172" Sep 10 20:03:01.231526 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.231492 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:03:01.232112 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.231562 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:03:01.318849 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.318814 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7769777fc8-jpjrf"] Sep 10 20:03:01.319036 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.318992 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7769777fc8-jpjrf" podUID="20f10edc-c486-4e27-a194-00b625329529" containerName="manager" containerID="cri-o://d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef" gracePeriod=30 Sep 10 20:03:01.563210 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.563184 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:03:01.710345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.710305 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") pod \"20f10edc-c486-4e27-a194-00b625329529\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " Sep 10 20:03:01.710528 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.710408 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/20f10edc-c486-4e27-a194-00b625329529-kube-api-access-dqghz\") pod \"20f10edc-c486-4e27-a194-00b625329529\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " Sep 10 20:03:01.710528 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.710437 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-metrics-certs\") pod \"20f10edc-c486-4e27-a194-00b625329529\" (UID: \"20f10edc-c486-4e27-a194-00b625329529\") " Sep 10 20:03:01.712776 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.712737 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "20f10edc-c486-4e27-a194-00b625329529" (UID: "20f10edc-c486-4e27-a194-00b625329529"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:03:01.712776 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.712761 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "20f10edc-c486-4e27-a194-00b625329529" (UID: "20f10edc-c486-4e27-a194-00b625329529"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:03:01.712960 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.712849 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f10edc-c486-4e27-a194-00b625329529-kube-api-access-dqghz" (OuterVolumeSpecName: "kube-api-access-dqghz") pod "20f10edc-c486-4e27-a194-00b625329529" (UID: "20f10edc-c486-4e27-a194-00b625329529"). InnerVolumeSpecName "kube-api-access-dqghz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:03:01.811903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.811866 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/20f10edc-c486-4e27-a194-00b625329529-kube-api-access-dqghz\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:03:01.811903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.811899 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:03:01.811903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:01.811909 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20f10edc-c486-4e27-a194-00b625329529-webhook-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:03:02.286164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.286126 2570 generic.go:356] "Generic (PLEG): container finished" podID="20f10edc-c486-4e27-a194-00b625329529" containerID="d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef" exitCode=0 Sep 10 20:03:02.286591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.286181 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-jpjrf" Sep 10 20:03:02.286591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.286214 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-jpjrf" event={"ID":"20f10edc-c486-4e27-a194-00b625329529","Type":"ContainerDied","Data":"d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef"} Sep 10 20:03:02.286591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.286269 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-jpjrf" event={"ID":"20f10edc-c486-4e27-a194-00b625329529","Type":"ContainerDied","Data":"cdf9cf708401baaa541a21073b5aa07ad9782c15e7206f5e02bea11f4976243b"} Sep 10 20:03:02.286591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.286292 2570 scope.go:117] "RemoveContainer" containerID="d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef" Sep 10 20:03:02.294805 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.294786 2570 scope.go:117] "RemoveContainer" containerID="d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef" Sep 10 20:03:02.295124 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:03:02.295098 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef\": container with ID starting with d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef not found: ID does not exist" containerID="d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef" Sep 10 20:03:02.295170 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.295139 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef"} err="failed to get container status \"d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef\": rpc error: code = NotFound desc = could not find container \"d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef\": container with ID starting with d8d9ee1c4750f909a97eeaa40a2e880961455926d808caa87404b573091207ef not found: ID does not exist" Sep 10 20:03:02.303827 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.303804 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7769777fc8-jpjrf"] Sep 10 20:03:02.308910 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:02.308885 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7769777fc8-jpjrf"] Sep 10 20:03:03.765770 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:03.765736 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f10edc-c486-4e27-a194-00b625329529" path="/var/lib/kubelet/pods/20f10edc-c486-4e27-a194-00b625329529/volumes" Sep 10 20:03:24.550455 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.550420 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6v58f"] Sep 10 20:03:24.550895 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.550820 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f10edc-c486-4e27-a194-00b625329529" containerName="manager" Sep 10 20:03:24.550895 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.550831 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f10edc-c486-4e27-a194-00b625329529" containerName="manager" Sep 10 20:03:24.550895 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.550894 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f10edc-c486-4e27-a194-00b625329529" containerName="manager" Sep 10 20:03:24.555572 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.555550 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.558032 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.558008 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lznxl\"" Sep 10 20:03:24.558165 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.558041 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 20:03:24.562587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.562564 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6v58f"] Sep 10 20:03:24.632552 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.632522 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcvth\" (UniqueName: \"kubernetes.io/projected/3e8f86ab-11fb-41d8-97e2-0b1077ea1e03-kube-api-access-rcvth\") pod \"payload-processing-5b54b7fbb7-6v58f\" (UID: \"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.632552 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.632556 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8f86ab-11fb-41d8-97e2-0b1077ea1e03-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6v58f\" (UID: \"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.733735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.733700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcvth\" (UniqueName: \"kubernetes.io/projected/3e8f86ab-11fb-41d8-97e2-0b1077ea1e03-kube-api-access-rcvth\") pod \"payload-processing-5b54b7fbb7-6v58f\" (UID: \"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.733735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.733734 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8f86ab-11fb-41d8-97e2-0b1077ea1e03-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6v58f\" (UID: \"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.734539 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.734517 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8f86ab-11fb-41d8-97e2-0b1077ea1e03-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6v58f\" (UID: \"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.741950 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.741921 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcvth\" (UniqueName: \"kubernetes.io/projected/3e8f86ab-11fb-41d8-97e2-0b1077ea1e03-kube-api-access-rcvth\") pod \"payload-processing-5b54b7fbb7-6v58f\" (UID: \"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:24.867174 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:24.867064 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:25.155354 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:25.155330 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6v58f"] Sep 10 20:03:25.158366 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:03:25.158313 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8f86ab_11fb_41d8_97e2_0b1077ea1e03.slice/crio-607f8d6dfed1b148c1a0058cb2f3ba873369489c5a68897404aa1b6ad2c7ef8a WatchSource:0}: Error finding container 607f8d6dfed1b148c1a0058cb2f3ba873369489c5a68897404aa1b6ad2c7ef8a: Status 404 returned error can't find the container with id 607f8d6dfed1b148c1a0058cb2f3ba873369489c5a68897404aa1b6ad2c7ef8a Sep 10 20:03:25.395794 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:25.395758 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" event={"ID":"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03","Type":"ContainerStarted","Data":"607f8d6dfed1b148c1a0058cb2f3ba873369489c5a68897404aa1b6ad2c7ef8a"} Sep 10 20:03:28.413728 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:28.413692 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" event={"ID":"3e8f86ab-11fb-41d8-97e2-0b1077ea1e03","Type":"ContainerStarted","Data":"ee2a942d4744eb003ed4720643dc6b7902960439e6b83aee15cf06681cf9c444"} Sep 10 20:03:28.414126 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:28.413759 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:28.437545 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:28.437489 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" podStartSLOduration=1.795702898 podStartE2EDuration="4.437469745s" totalImagesPullingTime="2.641766847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:03:28.436360387 +0000 UTC m=+577.157307715" watchObservedRunningTime="2026-09-10 20:03:28.437469745 +0000 UTC m=+577.158416865" Sep 10 20:03:39.419745 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:39.419716 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-6v58f" Sep 10 20:03:42.862232 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.862198 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf"] Sep 10 20:03:42.865778 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.865758 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:42.868250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.868228 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 20:03:42.869256 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.869234 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 20:03:42.869368 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.869241 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 20:03:42.869368 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.869339 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j5zgl\"" Sep 10 20:03:42.875560 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.875542 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf"] Sep 10 20:03:42.995690 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.995651 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:42.995845 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.995707 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:42.995845 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.995741 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:42.995845 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.995814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n849s\" (UniqueName: \"kubernetes.io/projected/18b97df0-df1c-41fc-9f06-ec00a01bb114-kube-api-access-n849s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:42.995950 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.995855 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18b97df0-df1c-41fc-9f06-ec00a01bb114-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:42.995950 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:42.995887 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.096583 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.096542 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.096756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.096605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n849s\" (UniqueName: \"kubernetes.io/projected/18b97df0-df1c-41fc-9f06-ec00a01bb114-kube-api-access-n849s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.096756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.096634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18b97df0-df1c-41fc-9f06-ec00a01bb114-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.096756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.096658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.096756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.096707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.096972 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.096796 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.097062 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.097039 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.097155 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.097045 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.097207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.097156 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.099168 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.099139 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18b97df0-df1c-41fc-9f06-ec00a01bb114-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.099272 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.099258 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18b97df0-df1c-41fc-9f06-ec00a01bb114-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.104470 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.104446 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n849s\" (UniqueName: \"kubernetes.io/projected/18b97df0-df1c-41fc-9f06-ec00a01bb114-kube-api-access-n849s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf\" (UID: \"18b97df0-df1c-41fc-9f06-ec00a01bb114\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.176980 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.176896 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:43.449922 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.449896 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf"] Sep 10 20:03:43.451703 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:03:43.451673 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b97df0_df1c_41fc_9f06_ec00a01bb114.slice/crio-061c93971b3d2adc13a3df2f28c1c1aca8991018715fdeaa665b42693121024d WatchSource:0}: Error finding container 061c93971b3d2adc13a3df2f28c1c1aca8991018715fdeaa665b42693121024d: Status 404 returned error can't find the container with id 061c93971b3d2adc13a3df2f28c1c1aca8991018715fdeaa665b42693121024d Sep 10 20:03:43.487391 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:43.487351 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" event={"ID":"18b97df0-df1c-41fc-9f06-ec00a01bb114","Type":"ContainerStarted","Data":"061c93971b3d2adc13a3df2f28c1c1aca8991018715fdeaa665b42693121024d"} Sep 10 20:03:45.499868 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:45.499833 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" event={"ID":"18b97df0-df1c-41fc-9f06-ec00a01bb114","Type":"ContainerStarted","Data":"6608f49eca602de41189686bae4284e8486c0ebf8171fdf1eb9ea26d38530e41"} Sep 10 20:03:45.516287 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:45.516226 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" podStartSLOduration=2.404298964 podStartE2EDuration="3.516206602s" totalImagesPullingTime="1.111907638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:03:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:03:45.515868423 +0000 UTC m=+594.236815545" watchObservedRunningTime="2026-09-10 20:03:45.516206602 +0000 UTC m=+594.237153722" Sep 10 20:03:53.177291 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:53.177252 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:53.177693 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:53.177328 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:53.189657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:53.189630 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:03:53.550740 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:03:53.550709 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf" Sep 10 20:04:05.532394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.532357 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl"] Sep 10 20:04:05.537594 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.537573 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.540889 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.540865 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 10 20:04:05.548786 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.548750 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl"] Sep 10 20:04:05.605465 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.605431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.605465 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.605470 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e40b7da6-5e82-4d1a-b44a-04098ba75439-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.605714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.605493 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.605714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.605507 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.605714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.605542 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.605714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.605573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m4sv\" (UniqueName: \"kubernetes.io/projected/e40b7da6-5e82-4d1a-b44a-04098ba75439-kube-api-access-9m4sv\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707087 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707033 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707273 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707102 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e40b7da6-5e82-4d1a-b44a-04098ba75439-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707273 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707136 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707273 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707273 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707194 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707273 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m4sv\" (UniqueName: \"kubernetes.io/projected/e40b7da6-5e82-4d1a-b44a-04098ba75439-kube-api-access-9m4sv\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707540 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707510 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707540 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707525 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.707686 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.707662 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.709507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.709475 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e40b7da6-5e82-4d1a-b44a-04098ba75439-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.709825 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.709808 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e40b7da6-5e82-4d1a-b44a-04098ba75439-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.715973 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.715943 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m4sv\" (UniqueName: \"kubernetes.io/projected/e40b7da6-5e82-4d1a-b44a-04098ba75439-kube-api-access-9m4sv\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl\" (UID: \"e40b7da6-5e82-4d1a-b44a-04098ba75439\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:05.851904 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:05.851819 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:06.123477 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:06.123452 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl"] Sep 10 20:04:06.126026 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:04:06.125995 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40b7da6_5e82_4d1a_b44a_04098ba75439.slice/crio-fe485a68b9184563154d50375dda457ec009cfac846f9580aaaaa08a1f11b6be WatchSource:0}: Error finding container fe485a68b9184563154d50375dda457ec009cfac846f9580aaaaa08a1f11b6be: Status 404 returned error can't find the container with id fe485a68b9184563154d50375dda457ec009cfac846f9580aaaaa08a1f11b6be Sep 10 20:04:06.601276 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:06.601229 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" event={"ID":"e40b7da6-5e82-4d1a-b44a-04098ba75439","Type":"ContainerStarted","Data":"c46ab37d96548d92c52c25a0d98019846fc2e49c85e04c88f676852d6a582649"} Sep 10 20:04:06.601757 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:06.601329 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" event={"ID":"e40b7da6-5e82-4d1a-b44a-04098ba75439","Type":"ContainerStarted","Data":"fe485a68b9184563154d50375dda457ec009cfac846f9580aaaaa08a1f11b6be"} Sep 10 20:04:06.622298 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:06.622248 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" podStartSLOduration=1.418040671 podStartE2EDuration="1.62223187s" totalImagesPullingTime="204.191199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:04:06.619897082 +0000 UTC m=+615.340844202" watchObservedRunningTime="2026-09-10 20:04:06.62223187 +0000 UTC m=+615.343178987" Sep 10 20:04:15.852912 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:15.852836 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:15.853345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:15.852915 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:15.865435 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:15.865412 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:04:16.660443 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:04:16.660419 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl" Sep 10 20:06:46.015425 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.015386 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz"] Sep 10 20:06:46.019094 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.019046 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.023809 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.023788 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6d8d62-openshift-default-dockercfg-4hgv4\"" Sep 10 20:06:46.037231 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.037201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz"] Sep 10 20:06:46.096359 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096319 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-data\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096359 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096355 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-envoy\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096382 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-token\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096399 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8d764-30b0-4afa-a9fc-4d192754e51f-istiod-ca-cert\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-socket\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096549 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-certs\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096734 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096602 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-credential-socket\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096734 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096628 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-podinfo\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.096734 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.096674 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vrfw\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-kube-api-access-9vrfw\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.197700 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.197660 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vrfw\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-kube-api-access-9vrfw\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.197885 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.197714 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-data\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.197885 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.197735 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-envoy\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.197885 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.197776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-token\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.197885 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.197799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8d764-30b0-4afa-a9fc-4d192754e51f-istiod-ca-cert\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198128 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.197950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-socket\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198128 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-certs\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198132 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-credential-socket\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198152 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-data\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-podinfo\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198386 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198364 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-socket\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198447 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198433 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-credential-socket\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198518 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198500 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-certs\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.198697 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.198674 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8d764-30b0-4afa-a9fc-4d192754e51f-istiod-ca-cert\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.200222 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.200201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-envoy\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.200534 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.200515 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-podinfo\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.205225 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.205201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vrfw\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-kube-api-access-9vrfw\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.205312 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.205196 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-token\") pod \"e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.330486 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.330393 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:46.606589 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.606564 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz"] Sep 10 20:06:46.608221 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:06:46.608195 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a8d764_30b0_4afa_a9fc_4d192754e51f.slice/crio-13220af37546b94320005bec1ee6a74ebcd58ee55fdc7455167bdd1a3d978619 WatchSource:0}: Error finding container 13220af37546b94320005bec1ee6a74ebcd58ee55fdc7455167bdd1a3d978619: Status 404 returned error can't find the container with id 13220af37546b94320005bec1ee6a74ebcd58ee55fdc7455167bdd1a3d978619 Sep 10 20:06:46.610422 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.610385 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:06:46.610510 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.610456 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:06:46.610510 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:46.610485 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:06:47.406915 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:47.406880 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" event={"ID":"73a8d764-30b0-4afa-a9fc-4d192754e51f","Type":"ContainerStarted","Data":"bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3"} Sep 10 20:06:47.407327 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:47.406930 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" event={"ID":"73a8d764-30b0-4afa-a9fc-4d192754e51f","Type":"ContainerStarted","Data":"13220af37546b94320005bec1ee6a74ebcd58ee55fdc7455167bdd1a3d978619"} Sep 10 20:06:47.427888 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:47.427838 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podStartSLOduration=2.427819748 podStartE2EDuration="2.427819748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:06:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:06:47.42683309 +0000 UTC m=+776.147780209" watchObservedRunningTime="2026-09-10 20:06:47.427819748 +0000 UTC m=+776.148766866" Sep 10 20:06:48.330986 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:48.330948 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:48.336185 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:48.336159 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:48.414176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:48.414142 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:48.414927 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:48.414909 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:06:51.349771 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.349732 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9"] Sep 10 20:06:51.353176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.353155 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.355508 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.355486 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6d8d62\"" Sep 10 20:06:51.355612 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.355492 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6d8d62-dockercfg-zs99b\"" Sep 10 20:06:51.363310 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.363284 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9"] Sep 10 20:06:51.449294 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.449258 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df003b36-363e-4a79-b7da-07c9a94f6405-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.449463 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.449342 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmwdx\" (UniqueName: \"kubernetes.io/projected/df003b36-363e-4a79-b7da-07c9a94f6405-kube-api-access-mmwdx\") pod \"payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.550381 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.550343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df003b36-363e-4a79-b7da-07c9a94f6405-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.550555 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.550394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwdx\" (UniqueName: \"kubernetes.io/projected/df003b36-363e-4a79-b7da-07c9a94f6405-kube-api-access-mmwdx\") pod \"payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.551118 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.551100 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df003b36-363e-4a79-b7da-07c9a94f6405-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.558595 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.558564 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmwdx\" (UniqueName: \"kubernetes.io/projected/df003b36-363e-4a79-b7da-07c9a94f6405-kube-api-access-mmwdx\") pod \"payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.666346 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.666271 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:51.937491 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:51.937368 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9"] Sep 10 20:06:51.940030 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:06:51.940003 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf003b36_363e_4a79_b7da_07c9a94f6405.slice/crio-3124b84d5ad36492cb188def2f485f04bc9074f201a94e982ad25f8c23c671d8 WatchSource:0}: Error finding container 3124b84d5ad36492cb188def2f485f04bc9074f201a94e982ad25f8c23c671d8: Status 404 returned error can't find the container with id 3124b84d5ad36492cb188def2f485f04bc9074f201a94e982ad25f8c23c671d8 Sep 10 20:06:52.435870 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.435832 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" event={"ID":"df003b36-363e-4a79-b7da-07c9a94f6405","Type":"ContainerStarted","Data":"e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9"} Sep 10 20:06:52.436316 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.435897 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" event={"ID":"df003b36-363e-4a79-b7da-07c9a94f6405","Type":"ContainerStarted","Data":"3124b84d5ad36492cb188def2f485f04bc9074f201a94e982ad25f8c23c671d8"} Sep 10 20:06:52.436316 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.435944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:06:52.451692 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.451638 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" podStartSLOduration=1.451619146 podStartE2EDuration="1.451619146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:06:52.451168088 +0000 UTC m=+781.172115216" watchObservedRunningTime="2026-09-10 20:06:52.451619146 +0000 UTC m=+781.172566266" Sep 10 20:06:52.645559 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.645519 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d"] Sep 10 20:06:52.649143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.649120 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.651938 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.651912 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-28a5f0-openshift-default-dockercfg-8gj8p\"" Sep 10 20:06:52.661783 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.661759 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d"] Sep 10 20:06:52.760463 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-data\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760466 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-envoy\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760496 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-token\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760516 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v65n\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-kube-api-access-7v65n\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760548 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-podinfo\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-socket\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760656 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istiod-ca-cert\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760731 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-credential-socket\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.760752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.760749 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-certs\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.861949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.861917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-credential-socket\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.861949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.861950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-certs\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862003 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-data\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-envoy\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-token\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862108 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v65n\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-kube-api-access-7v65n\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862165 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-podinfo\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-socket\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istiod-ca-cert\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862589 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862426 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-certs\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862589 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862526 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-credential-socket\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862889 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-data\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862971 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862874 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istiod-ca-cert\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.862971 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.862933 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-socket\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.864448 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.864420 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-envoy\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.864581 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.864561 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-podinfo\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.871820 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.871800 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-token\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.871926 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.871902 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v65n\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-kube-api-access-7v65n\") pod \"e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:52.963129 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:52.963097 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:53.236567 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.236323 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d"] Sep 10 20:06:53.238769 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:06:53.238741 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51ee1d0_47ee_46e6_8b00_2ace61a236c4.slice/crio-f686c3415cdc1b43948e010b3669152ad3902c6754dc3a7443a76b7ff43a0c1b WatchSource:0}: Error finding container f686c3415cdc1b43948e010b3669152ad3902c6754dc3a7443a76b7ff43a0c1b: Status 404 returned error can't find the container with id f686c3415cdc1b43948e010b3669152ad3902c6754dc3a7443a76b7ff43a0c1b Sep 10 20:06:53.241403 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.241365 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:06:53.241518 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.241430 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:06:53.241518 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.241462 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:06:53.442838 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.442786 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" event={"ID":"a51ee1d0-47ee-46e6-8b00-2ace61a236c4","Type":"ContainerStarted","Data":"df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6"} Sep 10 20:06:53.443343 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.442855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" event={"ID":"a51ee1d0-47ee-46e6-8b00-2ace61a236c4","Type":"ContainerStarted","Data":"f686c3415cdc1b43948e010b3669152ad3902c6754dc3a7443a76b7ff43a0c1b"} Sep 10 20:06:53.462736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.462677 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podStartSLOduration=1.46265651 podStartE2EDuration="1.46265651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:06:53.460284694 +0000 UTC m=+782.181231813" watchObservedRunningTime="2026-09-10 20:06:53.46265651 +0000 UTC m=+782.183603630" Sep 10 20:06:53.963195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:53.963162 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:54.968444 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:54.968410 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:55.453735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:55.453704 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:55.454801 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:55.454777 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:06:57.415910 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.415879 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm"] Sep 10 20:06:57.419181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.419164 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.421419 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.421399 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-28a5f0-dockercfg-kwh29\"" Sep 10 20:06:57.421504 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.421450 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-28a5f0\"" Sep 10 20:06:57.429819 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.429795 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm"] Sep 10 20:06:57.504266 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.504229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.504441 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.504292 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx4mt\" (UniqueName: \"kubernetes.io/projected/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-kube-api-access-xx4mt\") pod \"payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.605300 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.605264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.605494 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.605318 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx4mt\" (UniqueName: \"kubernetes.io/projected/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-kube-api-access-xx4mt\") pod \"payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.606006 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.605984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.616561 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.616537 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx4mt\" (UniqueName: \"kubernetes.io/projected/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-kube-api-access-xx4mt\") pod \"payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:57.730833 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:57.730744 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:58.008912 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:58.008879 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm"] Sep 10 20:06:58.012329 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:06:58.012302 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda389f9e4_1490_4d5d_a52e_a6cbfd6e4970.slice/crio-8cf31175c6e529f2f9c463fd17a6e5a8811a91a3dc44ac50ba36239157c1e9cb WatchSource:0}: Error finding container 8cf31175c6e529f2f9c463fd17a6e5a8811a91a3dc44ac50ba36239157c1e9cb: Status 404 returned error can't find the container with id 8cf31175c6e529f2f9c463fd17a6e5a8811a91a3dc44ac50ba36239157c1e9cb Sep 10 20:06:58.474034 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:58.473936 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" event={"ID":"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970","Type":"ContainerStarted","Data":"f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c"} Sep 10 20:06:58.474034 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:58.474007 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:06:58.474034 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:58.474031 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" event={"ID":"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970","Type":"ContainerStarted","Data":"8cf31175c6e529f2f9c463fd17a6e5a8811a91a3dc44ac50ba36239157c1e9cb"} Sep 10 20:06:58.492189 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:58.492133 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" podStartSLOduration=1.492111569 podStartE2EDuration="1.492111569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:06:58.491975501 +0000 UTC m=+787.212922620" watchObservedRunningTime="2026-09-10 20:06:58.492111569 +0000 UTC m=+787.213058689" Sep 10 20:06:59.209138 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.209098 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6"] Sep 10 20:06:59.212829 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.212804 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.215200 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.215176 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cde8cba8\"" Sep 10 20:06:59.215348 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.215229 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cde8cba8-dockercfg-njpw7\"" Sep 10 20:06:59.226482 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.226457 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6"] Sep 10 20:06:59.323286 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.323241 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/560ef692-4cec-439e-bb36-e8bc58769aa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.323500 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.323408 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prr9k\" (UniqueName: \"kubernetes.io/projected/560ef692-4cec-439e-bb36-e8bc58769aa2-kube-api-access-prr9k\") pod \"payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.424415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.424381 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prr9k\" (UniqueName: \"kubernetes.io/projected/560ef692-4cec-439e-bb36-e8bc58769aa2-kube-api-access-prr9k\") pod \"payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.424580 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.424473 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/560ef692-4cec-439e-bb36-e8bc58769aa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.425278 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.425257 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/560ef692-4cec-439e-bb36-e8bc58769aa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.432299 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.432276 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prr9k\" (UniqueName: \"kubernetes.io/projected/560ef692-4cec-439e-bb36-e8bc58769aa2-kube-api-access-prr9k\") pod \"payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.527520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.527494 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:06:59.830877 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:06:59.830739 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6"] Sep 10 20:06:59.833553 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:06:59.833525 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560ef692_4cec_439e_bb36_e8bc58769aa2.slice/crio-c493eaccb662d783248a84c2ed90309f0dddf5ad2ba884156e97803c26221dc1 WatchSource:0}: Error finding container c493eaccb662d783248a84c2ed90309f0dddf5ad2ba884156e97803c26221dc1: Status 404 returned error can't find the container with id c493eaccb662d783248a84c2ed90309f0dddf5ad2ba884156e97803c26221dc1 Sep 10 20:07:00.487572 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:00.487539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" event={"ID":"560ef692-4cec-439e-bb36-e8bc58769aa2","Type":"ContainerStarted","Data":"a33508e082a05168b4d10d2ce7f8051233c714bd9528a6c14cafba07b29a890c"} Sep 10 20:07:00.487785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:00.487592 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" event={"ID":"560ef692-4cec-439e-bb36-e8bc58769aa2","Type":"ContainerStarted","Data":"c493eaccb662d783248a84c2ed90309f0dddf5ad2ba884156e97803c26221dc1"} Sep 10 20:07:00.487785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:00.487622 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:07:00.503852 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:00.503781 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" podStartSLOduration=1.503765117 podStartE2EDuration="1.503765117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:07:00.501472296 +0000 UTC m=+789.222419413" watchObservedRunningTime="2026-09-10 20:07:00.503765117 +0000 UTC m=+789.224712233" Sep 10 20:07:03.445793 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:03.445756 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:07:07.527507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:07.527475 2570 generic.go:356] "Generic (PLEG): container finished" podID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerID="bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3" exitCode=0 Sep 10 20:07:07.527943 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:07.527530 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" event={"ID":"73a8d764-30b0-4afa-a9fc-4d192754e51f","Type":"ContainerDied","Data":"bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3"} Sep 10 20:07:07.527987 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:07.527947 2570 scope.go:117] "RemoveContainer" containerID="bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3" Sep 10 20:07:07.529080 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:07.529038 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:07.529137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:07.529124 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:07.529171 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:07.529163 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:08.533983 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:08.533951 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" event={"ID":"73a8d764-30b0-4afa-a9fc-4d192754e51f","Type":"ContainerStarted","Data":"3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787"} Sep 10 20:07:09.330648 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:09.330605 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:07:09.330823 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:09.330795 2570 patch_prober.go:28] interesting pod/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Sep 10 20:07:09.330912 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:09.330877 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Sep 10 20:07:09.484098 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:09.483617 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:07:10.331233 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:10.331201 2570 patch_prober.go:28] interesting pod/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Sep 10 20:07:10.331658 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:10.331269 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Sep 10 20:07:11.331113 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:11.331080 2570 patch_prober.go:28] interesting pod/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Sep 10 20:07:11.331352 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:11.331328 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Sep 10 20:07:11.497342 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:11.497307 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:07:12.330866 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:12.330828 2570 patch_prober.go:28] interesting pod/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Sep 10 20:07:12.331022 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:12.330879 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Sep 10 20:07:13.331632 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:13.331596 2570 patch_prober.go:28] interesting pod/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Sep 10 20:07:13.331998 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:13.331654 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Sep 10 20:07:14.335409 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:14.335323 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:07:14.335859 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:14.335636 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:07:14.336394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:14.336371 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:07:14.993911 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:14.993875 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5"] Sep 10 20:07:14.998102 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:14.998059 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.000674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.000648 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-39487c-openshift-default-dockercfg-z5q7n\"" Sep 10 20:07:15.008792 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.008761 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5"] Sep 10 20:07:15.118378 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-data\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118553 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118487 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-socket\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118553 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttfj4\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-kube-api-access-ttfj4\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118639 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1999064-b434-43fe-97fe-237be43d9b07-istiod-ca-cert\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118639 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118633 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1999064-b434-43fe-97fe-237be43d9b07-istio-podinfo\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118724 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-istio-token\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118724 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-credential-socket\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118810 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-envoy\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.118810 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.118785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-certs\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220161 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220126 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-envoy\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220161 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220163 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-certs\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-data\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220305 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-socket\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220330 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttfj4\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-kube-api-access-ttfj4\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1999064-b434-43fe-97fe-237be43d9b07-istiod-ca-cert\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220420 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1999064-b434-43fe-97fe-237be43d9b07-istio-podinfo\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220686 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-istio-token\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220686 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220481 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-credential-socket\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.220810 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220697 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-socket\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.221237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220931 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-certs\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.221237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220974 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-data\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.221237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.220980 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-credential-socket\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.221237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.221219 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1999064-b434-43fe-97fe-237be43d9b07-istiod-ca-cert\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.222506 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.222486 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-envoy\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.223004 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.222984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1999064-b434-43fe-97fe-237be43d9b07-istio-podinfo\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.228120 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.228092 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttfj4\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-kube-api-access-ttfj4\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.229248 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.229229 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-istio-token\") pod \"e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.312296 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.312194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:15.594235 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.594200 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5"] Sep 10 20:07:15.596415 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:07:15.596387 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1999064_b434_43fe_97fe_237be43d9b07.slice/crio-2457fa65758da01355d96f5ff57cab6f29591e085a55e49321e425b2abe9eebc WatchSource:0}: Error finding container 2457fa65758da01355d96f5ff57cab6f29591e085a55e49321e425b2abe9eebc: Status 404 returned error can't find the container with id 2457fa65758da01355d96f5ff57cab6f29591e085a55e49321e425b2abe9eebc Sep 10 20:07:15.598727 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.598686 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:15.598815 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.598782 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:15.598870 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:15.598827 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:16.601688 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.601601 2570 generic.go:356] "Generic (PLEG): container finished" podID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerID="df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6" exitCode=0 Sep 10 20:07:16.602234 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.601680 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" event={"ID":"a51ee1d0-47ee-46e6-8b00-2ace61a236c4","Type":"ContainerDied","Data":"df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6"} Sep 10 20:07:16.602307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.602231 2570 scope.go:117] "RemoveContainer" containerID="df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6" Sep 10 20:07:16.603285 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.603250 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" event={"ID":"b1999064-b434-43fe-97fe-237be43d9b07","Type":"ContainerStarted","Data":"765b292e8330f9637c4e83e03ee254d7b8bf95e205671975fa611e03439def81"} Sep 10 20:07:16.603402 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.603314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" event={"ID":"b1999064-b434-43fe-97fe-237be43d9b07","Type":"ContainerStarted","Data":"2457fa65758da01355d96f5ff57cab6f29591e085a55e49321e425b2abe9eebc"} Sep 10 20:07:16.603402 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.603366 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:16.603489 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.603436 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:16.603489 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.603470 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:16.645836 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:16.645699 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podStartSLOduration=2.645646762 podStartE2EDuration="2.645646762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:07:16.642361959 +0000 UTC m=+805.363309079" watchObservedRunningTime="2026-09-10 20:07:16.645646762 +0000 UTC m=+805.366593882" Sep 10 20:07:17.312779 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.312744 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:17.318401 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.318377 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:17.610314 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.610223 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" event={"ID":"a51ee1d0-47ee-46e6-8b00-2ace61a236c4","Type":"ContainerStarted","Data":"9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d"} Sep 10 20:07:17.610694 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.610485 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:17.611472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.611447 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:17.964183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.963863 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:07:17.964183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.963994 2570 patch_prober.go:28] interesting pod/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Sep 10 20:07:17.964183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:17.964039 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Sep 10 20:07:18.964039 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:18.964001 2570 patch_prober.go:28] interesting pod/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Sep 10 20:07:18.964521 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:18.964059 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Sep 10 20:07:19.964320 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:19.964280 2570 patch_prober.go:28] interesting pod/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Sep 10 20:07:19.964743 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:19.964343 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Sep 10 20:07:20.964095 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:20.963447 2570 patch_prober.go:28] interesting pod/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Sep 10 20:07:20.964095 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:20.963526 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Sep 10 20:07:21.552891 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.552847 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg"] Sep 10 20:07:21.562090 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.560697 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.564734 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.564460 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-39487c\"" Sep 10 20:07:21.568115 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.568088 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg"] Sep 10 20:07:21.598977 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.598947 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.599126 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.599063 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ks8h\" (UniqueName: \"kubernetes.io/projected/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-kube-api-access-4ks8h\") pod \"payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.700126 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.700062 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ks8h\" (UniqueName: \"kubernetes.io/projected/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-kube-api-access-4ks8h\") pod \"payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.700339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.700316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.701270 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.701246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.708715 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.708693 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ks8h\" (UniqueName: \"kubernetes.io/projected/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-kube-api-access-4ks8h\") pod \"payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.877255 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.877165 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:21.964157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.963635 2570 patch_prober.go:28] interesting pod/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Sep 10 20:07:21.964157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:21.963717 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Sep 10 20:07:22.151147 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.150465 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg"] Sep 10 20:07:22.171009 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:07:22.170978 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1350ed22_49e9_4c1b_875e_2a54e3ca4f6c.slice/crio-300f916d95c0640224f758aeed2e5a5f6e6ed330598741974294f021d101c346 WatchSource:0}: Error finding container 300f916d95c0640224f758aeed2e5a5f6e6ed330598741974294f021d101c346: Status 404 returned error can't find the container with id 300f916d95c0640224f758aeed2e5a5f6e6ed330598741974294f021d101c346 Sep 10 20:07:22.203118 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.203095 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh"] Sep 10 20:07:22.209155 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.208127 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:22.212263 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.212239 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:07:22.212456 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.212438 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:07:22.214767 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.214743 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh"] Sep 10 20:07:22.304083 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.304036 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3159a8-eeaa-456d-9313-2b2af6ef10d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh\" (UID: \"bd3159a8-eeaa-456d-9313-2b2af6ef10d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:22.404591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.404506 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3159a8-eeaa-456d-9313-2b2af6ef10d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh\" (UID: \"bd3159a8-eeaa-456d-9313-2b2af6ef10d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:22.405204 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.405183 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3159a8-eeaa-456d-9313-2b2af6ef10d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh\" (UID: \"bd3159a8-eeaa-456d-9313-2b2af6ef10d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:22.565318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.565283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:22.674971 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.673876 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" event={"ID":"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c","Type":"ContainerStarted","Data":"ac76307cb9cdd53bd773e3b1efab834e3788bcd5cbb23976ff5265dc21eb06e6"} Sep 10 20:07:22.674971 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.673942 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" event={"ID":"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c","Type":"ContainerStarted","Data":"300f916d95c0640224f758aeed2e5a5f6e6ed330598741974294f021d101c346"} Sep 10 20:07:22.674971 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.674758 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:22.693624 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.693563 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" podStartSLOduration=1.693548976 podStartE2EDuration="1.693548976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:07:22.692352863 +0000 UTC m=+811.413299982" watchObservedRunningTime="2026-09-10 20:07:22.693548976 +0000 UTC m=+811.414496094" Sep 10 20:07:22.848670 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.848641 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh"] Sep 10 20:07:22.854554 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:22.854482 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 20:07:23.111871 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:23.111768 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:07:23.112226 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:23.112188 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:07:23.113567 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:23.113534 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:07:23.680229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:23.680193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" event={"ID":"bd3159a8-eeaa-456d-9313-2b2af6ef10d4","Type":"ContainerStarted","Data":"691412a21b99ede4de8fe3a58f7c032b44f8d45297e92725a56f104f1f304a4e"} Sep 10 20:07:25.694146 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:25.694110 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" event={"ID":"bd3159a8-eeaa-456d-9313-2b2af6ef10d4","Type":"ContainerStarted","Data":"eca11ec0eea20cdcfee5c4c2020033f0204ab7108d9abb1438c24ff37a2ea630"} Sep 10 20:07:25.708947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:25.708895 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" podStartSLOduration=1.336784063 podStartE2EDuration="3.708877304s" totalImagesPullingTime="2.372093241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:07:25.708260428 +0000 UTC m=+814.429207551" watchObservedRunningTime="2026-09-10 20:07:25.708877304 +0000 UTC m=+814.429824423" Sep 10 20:07:26.700834 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:26.700799 2570 generic.go:356] "Generic (PLEG): container finished" podID="bd3159a8-eeaa-456d-9313-2b2af6ef10d4" containerID="eca11ec0eea20cdcfee5c4c2020033f0204ab7108d9abb1438c24ff37a2ea630" exitCode=0 Sep 10 20:07:26.701255 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:26.700891 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" event={"ID":"bd3159a8-eeaa-456d-9313-2b2af6ef10d4","Type":"ContainerDied","Data":"eca11ec0eea20cdcfee5c4c2020033f0204ab7108d9abb1438c24ff37a2ea630"} Sep 10 20:07:27.845130 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:27.845106 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:27.964893 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:27.964807 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3159a8-eeaa-456d-9313-2b2af6ef10d4-maas-api-service-ca\") pod \"bd3159a8-eeaa-456d-9313-2b2af6ef10d4\" (UID: \"bd3159a8-eeaa-456d-9313-2b2af6ef10d4\") " Sep 10 20:07:27.965218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:27.965191 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd3159a8-eeaa-456d-9313-2b2af6ef10d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bd3159a8-eeaa-456d-9313-2b2af6ef10d4" (UID: "bd3159a8-eeaa-456d-9313-2b2af6ef10d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:07:28.065764 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:28.065733 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3159a8-eeaa-456d-9313-2b2af6ef10d4-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:07:28.712694 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:28.712661 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" Sep 10 20:07:28.712878 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:28.712688 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh" event={"ID":"bd3159a8-eeaa-456d-9313-2b2af6ef10d4","Type":"ContainerDied","Data":"691412a21b99ede4de8fe3a58f7c032b44f8d45297e92725a56f104f1f304a4e"} Sep 10 20:07:28.712878 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:28.712734 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="691412a21b99ede4de8fe3a58f7c032b44f8d45297e92725a56f104f1f304a4e" Sep 10 20:07:34.689462 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:34.689422 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:07:37.769215 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:37.769177 2570 generic.go:356] "Generic (PLEG): container finished" podID="b1999064-b434-43fe-97fe-237be43d9b07" containerID="765b292e8330f9637c4e83e03ee254d7b8bf95e205671975fa611e03439def81" exitCode=0 Sep 10 20:07:37.769676 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:37.769246 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" event={"ID":"b1999064-b434-43fe-97fe-237be43d9b07","Type":"ContainerDied","Data":"765b292e8330f9637c4e83e03ee254d7b8bf95e205671975fa611e03439def81"} Sep 10 20:07:37.769676 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:37.769650 2570 scope.go:117] "RemoveContainer" containerID="765b292e8330f9637c4e83e03ee254d7b8bf95e205671975fa611e03439def81" Sep 10 20:07:37.770709 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:37.770674 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:37.770772 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:37.770754 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:37.770807 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:37.770794 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:07:38.546009 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.545970 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6"] Sep 10 20:07:38.546313 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.546281 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" podUID="560ef692-4cec-439e-bb36-e8bc58769aa2" containerName="payload-pre-processing" containerID="cri-o://a33508e082a05168b4d10d2ce7f8051233c714bd9528a6c14cafba07b29a890c" gracePeriod=30 Sep 10 20:07:38.707132 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.707094 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh"] Sep 10 20:07:38.713592 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.713564 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cde8cba8-4cjnh"] Sep 10 20:07:38.785385 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.785230 2570 generic.go:356] "Generic (PLEG): container finished" podID="560ef692-4cec-439e-bb36-e8bc58769aa2" containerID="a33508e082a05168b4d10d2ce7f8051233c714bd9528a6c14cafba07b29a890c" exitCode=0 Sep 10 20:07:38.785385 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.785317 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" event={"ID":"560ef692-4cec-439e-bb36-e8bc58769aa2","Type":"ContainerDied","Data":"a33508e082a05168b4d10d2ce7f8051233c714bd9528a6c14cafba07b29a890c"} Sep 10 20:07:38.789671 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.788702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" event={"ID":"b1999064-b434-43fe-97fe-237be43d9b07","Type":"ContainerStarted","Data":"83d8cb25fe8014fab516625b70bfe2590f61cba9f760f1fa85f3ffb0de661b31"} Sep 10 20:07:38.879991 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.879967 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:07:38.976685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.976641 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/560ef692-4cec-439e-bb36-e8bc58769aa2-plugins-config-volume\") pod \"560ef692-4cec-439e-bb36-e8bc58769aa2\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " Sep 10 20:07:38.976866 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.976809 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prr9k\" (UniqueName: \"kubernetes.io/projected/560ef692-4cec-439e-bb36-e8bc58769aa2-kube-api-access-prr9k\") pod \"560ef692-4cec-439e-bb36-e8bc58769aa2\" (UID: \"560ef692-4cec-439e-bb36-e8bc58769aa2\") " Sep 10 20:07:38.977085 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.977041 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/560ef692-4cec-439e-bb36-e8bc58769aa2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "560ef692-4cec-439e-bb36-e8bc58769aa2" (UID: "560ef692-4cec-439e-bb36-e8bc58769aa2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:07:38.977501 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.977480 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/560ef692-4cec-439e-bb36-e8bc58769aa2-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:07:38.979226 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:38.979208 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/560ef692-4cec-439e-bb36-e8bc58769aa2-kube-api-access-prr9k" (OuterVolumeSpecName: "kube-api-access-prr9k") pod "560ef692-4cec-439e-bb36-e8bc58769aa2" (UID: "560ef692-4cec-439e-bb36-e8bc58769aa2"). InnerVolumeSpecName "kube-api-access-prr9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:07:39.078779 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.078739 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prr9k\" (UniqueName: \"kubernetes.io/projected/560ef692-4cec-439e-bb36-e8bc58769aa2-kube-api-access-prr9k\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:07:39.313239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.313203 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:39.313418 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.313343 2570 patch_prober.go:28] interesting pod/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Sep 10 20:07:39.313418 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.313388 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Sep 10 20:07:39.777122 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.775889 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd3159a8-eeaa-456d-9313-2b2af6ef10d4" path="/var/lib/kubelet/pods/bd3159a8-eeaa-456d-9313-2b2af6ef10d4/volumes" Sep 10 20:07:39.799599 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.799574 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" Sep 10 20:07:39.799989 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.799606 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6" event={"ID":"560ef692-4cec-439e-bb36-e8bc58769aa2","Type":"ContainerDied","Data":"c493eaccb662d783248a84c2ed90309f0dddf5ad2ba884156e97803c26221dc1"} Sep 10 20:07:39.799989 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.799668 2570 scope.go:117] "RemoveContainer" containerID="a33508e082a05168b4d10d2ce7f8051233c714bd9528a6c14cafba07b29a890c" Sep 10 20:07:39.815998 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.815970 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6"] Sep 10 20:07:39.819625 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:39.819605 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cde8cba8-745cb8f8db-szln6"] Sep 10 20:07:40.313543 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:40.313509 2570 patch_prober.go:28] interesting pod/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Sep 10 20:07:40.313714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:40.313573 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Sep 10 20:07:41.312712 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:41.312677 2570 patch_prober.go:28] interesting pod/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Sep 10 20:07:41.313114 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:41.312731 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Sep 10 20:07:41.772768 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:41.772732 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="560ef692-4cec-439e-bb36-e8bc58769aa2" path="/var/lib/kubelet/pods/560ef692-4cec-439e-bb36-e8bc58769aa2/volumes" Sep 10 20:07:42.313309 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:42.313272 2570 patch_prober.go:28] interesting pod/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Sep 10 20:07:42.313704 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:42.313338 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Sep 10 20:07:43.313498 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.313465 2570 patch_prober.go:28] interesting pod/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Sep 10 20:07:43.313855 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.313536 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Sep 10 20:07:43.588930 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.588849 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs"] Sep 10 20:07:43.589460 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.589443 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="560ef692-4cec-439e-bb36-e8bc58769aa2" containerName="payload-pre-processing" Sep 10 20:07:43.589460 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.589461 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="560ef692-4cec-439e-bb36-e8bc58769aa2" containerName="payload-pre-processing" Sep 10 20:07:43.589573 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.589502 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3159a8-eeaa-456d-9313-2b2af6ef10d4" containerName="revoke-keys" Sep 10 20:07:43.589573 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.589512 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3159a8-eeaa-456d-9313-2b2af6ef10d4" containerName="revoke-keys" Sep 10 20:07:43.589651 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.589614 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="560ef692-4cec-439e-bb36-e8bc58769aa2" containerName="payload-pre-processing" Sep 10 20:07:43.589651 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.589624 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3159a8-eeaa-456d-9313-2b2af6ef10d4" containerName="revoke-keys" Sep 10 20:07:43.594035 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.594014 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.597125 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.597105 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-f563295c-dockercfg-jds6h\"" Sep 10 20:07:43.597229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.597170 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-f563295c\"" Sep 10 20:07:43.605501 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.605478 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs"] Sep 10 20:07:43.625133 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.625102 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6"] Sep 10 20:07:43.628568 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.628551 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.645092 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.643790 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6"] Sep 10 20:07:43.728768 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.728732 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xppt6\" (UniqueName: \"kubernetes.io/projected/9371a25c-211b-4ab6-8896-bdb25a14684c-kube-api-access-xppt6\") pod \"payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.728978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.728826 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dltc\" (UniqueName: \"kubernetes.io/projected/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-kube-api-access-2dltc\") pod \"payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.728978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.728870 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9371a25c-211b-4ab6-8896-bdb25a14684c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.728978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.728940 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.830439 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.830407 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xppt6\" (UniqueName: \"kubernetes.io/projected/9371a25c-211b-4ab6-8896-bdb25a14684c-kube-api-access-xppt6\") pod \"payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.830641 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.830513 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dltc\" (UniqueName: \"kubernetes.io/projected/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-kube-api-access-2dltc\") pod \"payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.830641 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.830572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9371a25c-211b-4ab6-8896-bdb25a14684c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.830892 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.830860 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.831327 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.831310 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9371a25c-211b-4ab6-8896-bdb25a14684c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.831453 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.831435 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.842551 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.842473 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dltc\" (UniqueName: \"kubernetes.io/projected/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-kube-api-access-2dltc\") pod \"payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:43.842551 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.842533 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xppt6\" (UniqueName: \"kubernetes.io/projected/9371a25c-211b-4ab6-8896-bdb25a14684c-kube-api-access-xppt6\") pod \"payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.905903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.905865 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:43.960559 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:43.960285 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:44.226626 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.226600 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs"] Sep 10 20:07:44.228512 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:07:44.228479 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9371a25c_211b_4ab6_8896_bdb25a14684c.slice/crio-fc2202088f2430499b2b8aca10741f19a1088967073bc357dbbdd70cf16424ac WatchSource:0}: Error finding container fc2202088f2430499b2b8aca10741f19a1088967073bc357dbbdd70cf16424ac: Status 404 returned error can't find the container with id fc2202088f2430499b2b8aca10741f19a1088967073bc357dbbdd70cf16424ac Sep 10 20:07:44.260407 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.260378 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6"] Sep 10 20:07:44.263390 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:07:44.263346 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a83ee5e_9e07_46e0_a1bc_2e9a57b051b6.slice/crio-01446fbcbfd85c5ba54c302477acc10e528565fc2026ebac6b8e80053a5e91ba WatchSource:0}: Error finding container 01446fbcbfd85c5ba54c302477acc10e528565fc2026ebac6b8e80053a5e91ba: Status 404 returned error can't find the container with id 01446fbcbfd85c5ba54c302477acc10e528565fc2026ebac6b8e80053a5e91ba Sep 10 20:07:44.577151 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.577117 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:44.578151 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.578121 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:44.579856 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.579837 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:07:44.828579 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.828483 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" event={"ID":"9371a25c-211b-4ab6-8896-bdb25a14684c","Type":"ContainerStarted","Data":"f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0"} Sep 10 20:07:44.828579 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.828568 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" event={"ID":"9371a25c-211b-4ab6-8896-bdb25a14684c","Type":"ContainerStarted","Data":"fc2202088f2430499b2b8aca10741f19a1088967073bc357dbbdd70cf16424ac"} Sep 10 20:07:44.828821 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.828611 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:07:44.830313 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.830283 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" event={"ID":"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6","Type":"ContainerStarted","Data":"95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c"} Sep 10 20:07:44.830452 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.830331 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" event={"ID":"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6","Type":"ContainerStarted","Data":"01446fbcbfd85c5ba54c302477acc10e528565fc2026ebac6b8e80053a5e91ba"} Sep 10 20:07:44.830523 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.830480 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:44.845253 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.845200 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" podStartSLOduration=1.845180144 podStartE2EDuration="1.845180144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:07:44.843299416 +0000 UTC m=+833.564246538" watchObservedRunningTime="2026-09-10 20:07:44.845180144 +0000 UTC m=+833.566127265" Sep 10 20:07:44.860615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:44.860564 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" podStartSLOduration=1.860543434 podStartE2EDuration="1.860543434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:07:44.858185082 +0000 UTC m=+833.579132197" watchObservedRunningTime="2026-09-10 20:07:44.860543434 +0000 UTC m=+833.581490553" Sep 10 20:07:55.838308 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:55.838277 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:07:55.838822 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:07:55.838604 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:08:23.665357 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:23.665046 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs"] Sep 10 20:08:23.665357 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:23.665313 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" podUID="9371a25c-211b-4ab6-8896-bdb25a14684c" containerName="payload-processing" containerID="cri-o://f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0" gracePeriod=30 Sep 10 20:08:23.671630 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:23.671593 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6"] Sep 10 20:08:23.673065 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:23.673015 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" podUID="8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" containerName="payload-pre-processing" containerID="cri-o://95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c" gracePeriod=30 Sep 10 20:08:23.989925 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:23.989900 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:08:23.993892 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:23.993863 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:08:24.056479 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.056442 2570 generic.go:356] "Generic (PLEG): container finished" podID="8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" containerID="95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c" exitCode=0 Sep 10 20:08:24.056814 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.056565 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" Sep 10 20:08:24.056993 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.056590 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" event={"ID":"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6","Type":"ContainerDied","Data":"95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c"} Sep 10 20:08:24.057471 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.057441 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6" event={"ID":"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6","Type":"ContainerDied","Data":"01446fbcbfd85c5ba54c302477acc10e528565fc2026ebac6b8e80053a5e91ba"} Sep 10 20:08:24.059290 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.059226 2570 generic.go:356] "Generic (PLEG): container finished" podID="9371a25c-211b-4ab6-8896-bdb25a14684c" containerID="f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0" exitCode=0 Sep 10 20:08:24.059571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.059549 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" event={"ID":"9371a25c-211b-4ab6-8896-bdb25a14684c","Type":"ContainerDied","Data":"f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0"} Sep 10 20:08:24.059940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.059920 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" event={"ID":"9371a25c-211b-4ab6-8896-bdb25a14684c","Type":"ContainerDied","Data":"fc2202088f2430499b2b8aca10741f19a1088967073bc357dbbdd70cf16424ac"} Sep 10 20:08:24.060247 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.060229 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs" Sep 10 20:08:24.060666 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.057490 2570 scope.go:117] "RemoveContainer" containerID="95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c" Sep 10 20:08:24.073971 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.073699 2570 scope.go:117] "RemoveContainer" containerID="95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c" Sep 10 20:08:24.075012 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:08:24.074968 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c\": container with ID starting with 95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c not found: ID does not exist" containerID="95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c" Sep 10 20:08:24.075140 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.075008 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c"} err="failed to get container status \"95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c\": rpc error: code = NotFound desc = could not find container \"95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c\": container with ID starting with 95d78e29daf18a4d7015b3f67ec368814919c3efbe9e07403ba456284361d98c not found: ID does not exist" Sep 10 20:08:24.075140 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.075032 2570 scope.go:117] "RemoveContainer" containerID="f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0" Sep 10 20:08:24.090548 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.090523 2570 scope.go:117] "RemoveContainer" containerID="f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0" Sep 10 20:08:24.090962 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:08:24.090936 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0\": container with ID starting with f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0 not found: ID does not exist" containerID="f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0" Sep 10 20:08:24.091055 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.090970 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0"} err="failed to get container status \"f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0\": rpc error: code = NotFound desc = could not find container \"f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0\": container with ID starting with f570ab395ca7fd14b1c7748427a09989c6553ecca2db7d95b6b25c024b0e50d0 not found: ID does not exist" Sep 10 20:08:24.135474 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.135452 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-plugins-config-volume\") pod \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " Sep 10 20:08:24.135581 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.135557 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dltc\" (UniqueName: \"kubernetes.io/projected/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-kube-api-access-2dltc\") pod \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\" (UID: \"8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6\") " Sep 10 20:08:24.135656 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.135581 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xppt6\" (UniqueName: \"kubernetes.io/projected/9371a25c-211b-4ab6-8896-bdb25a14684c-kube-api-access-xppt6\") pod \"9371a25c-211b-4ab6-8896-bdb25a14684c\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " Sep 10 20:08:24.135656 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.135612 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9371a25c-211b-4ab6-8896-bdb25a14684c-plugins-config-volume\") pod \"9371a25c-211b-4ab6-8896-bdb25a14684c\" (UID: \"9371a25c-211b-4ab6-8896-bdb25a14684c\") " Sep 10 20:08:24.135859 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.135833 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" (UID: "8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:08:24.136012 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.135988 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9371a25c-211b-4ab6-8896-bdb25a14684c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9371a25c-211b-4ab6-8896-bdb25a14684c" (UID: "9371a25c-211b-4ab6-8896-bdb25a14684c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:08:24.137711 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.137691 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9371a25c-211b-4ab6-8896-bdb25a14684c-kube-api-access-xppt6" (OuterVolumeSpecName: "kube-api-access-xppt6") pod "9371a25c-211b-4ab6-8896-bdb25a14684c" (UID: "9371a25c-211b-4ab6-8896-bdb25a14684c"). InnerVolumeSpecName "kube-api-access-xppt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:08:24.137986 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.137966 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-kube-api-access-2dltc" (OuterVolumeSpecName: "kube-api-access-2dltc") pod "8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" (UID: "8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6"). InnerVolumeSpecName "kube-api-access-2dltc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:08:24.237299 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.237203 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:08:24.237299 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.237241 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dltc\" (UniqueName: \"kubernetes.io/projected/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6-kube-api-access-2dltc\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:08:24.237299 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.237251 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xppt6\" (UniqueName: \"kubernetes.io/projected/9371a25c-211b-4ab6-8896-bdb25a14684c-kube-api-access-xppt6\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:08:24.237299 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.237259 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9371a25c-211b-4ab6-8896-bdb25a14684c-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:08:24.379782 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.379754 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6"] Sep 10 20:08:24.383750 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.383728 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f563295c-6f75c9dc94-lx4s6"] Sep 10 20:08:24.397082 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.397049 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs"] Sep 10 20:08:24.401118 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.401094 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f563295c-54589f56b-zvfqs"] Sep 10 20:08:24.722956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.722919 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5"] Sep 10 20:08:24.723415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.723392 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" containerName="payload-pre-processing" Sep 10 20:08:24.723415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.723409 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" containerName="payload-pre-processing" Sep 10 20:08:24.723556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.723432 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9371a25c-211b-4ab6-8896-bdb25a14684c" containerName="payload-processing" Sep 10 20:08:24.723556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.723438 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9371a25c-211b-4ab6-8896-bdb25a14684c" containerName="payload-processing" Sep 10 20:08:24.723556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.723507 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" containerName="payload-pre-processing" Sep 10 20:08:24.723556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.723519 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9371a25c-211b-4ab6-8896-bdb25a14684c" containerName="payload-processing" Sep 10 20:08:24.728195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.728171 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.730760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.730741 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-468cfd8a-openshift-default-dockercfg-cttlh\"" Sep 10 20:08:24.740770 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.740747 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5"] Sep 10 20:08:24.843158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843116 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-socket\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843163 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-podinfo\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843183 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl42r\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-kube-api-access-gl42r\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843236 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-data\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843327 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istiod-ca-cert\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843382 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-envoy\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843554 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-credential-socket\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843554 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843445 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-token\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.843554 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.843515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-certs\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944446 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-socket\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944481 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-podinfo\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl42r\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-kube-api-access-gl42r\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-data\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944880 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istiod-ca-cert\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944880 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-envoy\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944880 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944622 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-credential-socket\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944880 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-token\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.944880 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944712 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-certs\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.945183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-socket\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.945183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.944937 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-data\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.945183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.945100 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-credential-socket\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.945183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.945177 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-certs\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.945412 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.945393 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istiod-ca-cert\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.947015 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.946996 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-envoy\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.947329 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.947308 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-podinfo\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.952324 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.952303 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl42r\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-kube-api-access-gl42r\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:24.952423 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:24.952373 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-token\") pod \"e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:25.040413 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.040381 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:25.328467 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.328440 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5"] Sep 10 20:08:25.329999 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:08:25.329976 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3a3a3e_a5a3_45f3_a210_443b64fddd2e.slice/crio-db4996f42d95dd450b487c8de97617532abf77b316f5f7715bcc9a709b0fd215 WatchSource:0}: Error finding container db4996f42d95dd450b487c8de97617532abf77b316f5f7715bcc9a709b0fd215: Status 404 returned error can't find the container with id db4996f42d95dd450b487c8de97617532abf77b316f5f7715bcc9a709b0fd215 Sep 10 20:08:25.332181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.332153 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:08:25.332249 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.332217 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:08:25.332286 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.332248 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:08:25.767438 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.767405 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6" path="/var/lib/kubelet/pods/8a83ee5e-9e07-46e0-a1bc-2e9a57b051b6/volumes" Sep 10 20:08:25.767904 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:25.767823 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9371a25c-211b-4ab6-8896-bdb25a14684c" path="/var/lib/kubelet/pods/9371a25c-211b-4ab6-8896-bdb25a14684c/volumes" Sep 10 20:08:26.076808 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:26.076715 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" event={"ID":"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e","Type":"ContainerStarted","Data":"71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d"} Sep 10 20:08:26.076808 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:26.076774 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" event={"ID":"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e","Type":"ContainerStarted","Data":"db4996f42d95dd450b487c8de97617532abf77b316f5f7715bcc9a709b0fd215"} Sep 10 20:08:26.101506 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:26.101449 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podStartSLOduration=2.101433703 podStartE2EDuration="2.101433703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:08:26.098859514 +0000 UTC m=+874.819806646" watchObservedRunningTime="2026-09-10 20:08:26.101433703 +0000 UTC m=+874.822380821" Sep 10 20:08:27.040828 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:27.040789 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:27.046892 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:27.046867 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:27.082117 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:27.082081 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:27.082963 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:27.082943 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:52.229735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:52.229637 2570 generic.go:356] "Generic (PLEG): container finished" podID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerID="71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d" exitCode=0 Sep 10 20:08:52.229735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:52.229714 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" event={"ID":"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e","Type":"ContainerDied","Data":"71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d"} Sep 10 20:08:52.230544 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:52.230195 2570 scope.go:117] "RemoveContainer" containerID="71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d" Sep 10 20:08:52.231219 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:52.231192 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:08:52.231310 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:52.231293 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:08:52.231346 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:52.231335 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:08:53.238831 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:53.237836 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" event={"ID":"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e","Type":"ContainerStarted","Data":"6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199"} Sep 10 20:08:54.041364 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:54.041331 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:54.042167 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:54.042134 2570 patch_prober.go:28] interesting pod/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 10 20:08:54.042396 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:54.042371 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 10 20:08:55.041445 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:55.041402 2570 patch_prober.go:28] interesting pod/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 10 20:08:55.041954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:55.041472 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 10 20:08:56.041595 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:56.041555 2570 patch_prober.go:28] interesting pod/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 10 20:08:56.042233 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:56.042200 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 10 20:08:57.046524 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:57.046479 2570 patch_prober.go:28] interesting pod/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 10 20:08:57.047153 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:57.047116 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 10 20:08:57.082814 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:57.082782 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:58.041361 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:58.041323 2570 patch_prober.go:28] interesting pod/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 10 20:08:58.041682 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:58.041642 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 10 20:08:59.044826 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:59.044797 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:59.045752 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:59.045728 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:08:59.624083 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:59.624033 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm"] Sep 10 20:08:59.624345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:59.624317 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" podUID="a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" containerName="payload-pre-processing" containerID="cri-o://f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c" gracePeriod=30 Sep 10 20:08:59.911686 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:08:59.911660 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:09:00.045933 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.045901 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx4mt\" (UniqueName: \"kubernetes.io/projected/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-kube-api-access-xx4mt\") pod \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " Sep 10 20:09:00.046384 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.046005 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-plugins-config-volume\") pod \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\" (UID: \"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970\") " Sep 10 20:09:00.046467 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.046440 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" (UID: "a389f9e4-1490-4d5d-a52e-a6cbfd6e4970"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:09:00.048038 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.048014 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-kube-api-access-xx4mt" (OuterVolumeSpecName: "kube-api-access-xx4mt") pod "a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" (UID: "a389f9e4-1490-4d5d-a52e-a6cbfd6e4970"). InnerVolumeSpecName "kube-api-access-xx4mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:09:00.146970 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.146888 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:00.146970 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.146917 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx4mt\" (UniqueName: \"kubernetes.io/projected/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970-kube-api-access-xx4mt\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:00.286490 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.286457 2570 generic.go:356] "Generic (PLEG): container finished" podID="a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" containerID="f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c" exitCode=0 Sep 10 20:09:00.286654 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.286502 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" Sep 10 20:09:00.286654 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.286514 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" event={"ID":"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970","Type":"ContainerDied","Data":"f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c"} Sep 10 20:09:00.286654 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.286568 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm" event={"ID":"a389f9e4-1490-4d5d-a52e-a6cbfd6e4970","Type":"ContainerDied","Data":"8cf31175c6e529f2f9c463fd17a6e5a8811a91a3dc44ac50ba36239157c1e9cb"} Sep 10 20:09:00.286654 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.286590 2570 scope.go:117] "RemoveContainer" containerID="f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c" Sep 10 20:09:00.295706 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.295687 2570 scope.go:117] "RemoveContainer" containerID="f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c" Sep 10 20:09:00.295969 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:09:00.295952 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c\": container with ID starting with f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c not found: ID does not exist" containerID="f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c" Sep 10 20:09:00.296040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.295978 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c"} err="failed to get container status \"f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c\": rpc error: code = NotFound desc = could not find container \"f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c\": container with ID starting with f7d2f1b441ff5acb39aa71709146aaf62869b1463afbe5e1bee50794e83a6c8c not found: ID does not exist" Sep 10 20:09:00.318742 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.318708 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm"] Sep 10 20:09:00.326401 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.326381 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-28a5f0-5665dd447d-qf8dm"] Sep 10 20:09:00.883284 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.883253 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5"] Sep 10 20:09:00.884627 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:00.883952 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" containerID="cri-o://6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199" gracePeriod=30 Sep 10 20:09:01.770815 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:01.770781 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" path="/var/lib/kubelet/pods/a389f9e4-1490-4d5d-a52e-a6cbfd6e4970/volumes" Sep 10 20:09:06.168305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.168244 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:09:06.199474 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199441 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istiod-ca-cert\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.199617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199486 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl42r\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-kube-api-access-gl42r\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.199617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199538 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-podinfo\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.199617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199573 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-socket\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.199826 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199795 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-envoy\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.199907 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199828 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:09:06.199907 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199854 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:06.199907 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199859 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-credential-socket\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.200064 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199907 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-data\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.200064 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.199985 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-certs\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.200064 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200035 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-token\") pod \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\" (UID: \"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e\") " Sep 10 20:09:06.200250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200090 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:06.200250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200212 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:06.200363 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200312 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:06.200514 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200491 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.200597 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200517 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.200597 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200531 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.200597 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200544 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.200597 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.200556 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.201993 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.201967 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:06.202266 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.202241 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:09:06.202377 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.202281 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-kube-api-access-gl42r" (OuterVolumeSpecName: "kube-api-access-gl42r") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "kube-api-access-gl42r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:09:06.202464 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.202450 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" (UID: "ce3a3a3e-a5a3-45f3-a210-443b64fddd2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:09:06.297798 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.297766 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6"] Sep 10 20:09:06.298212 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298199 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" containerName="payload-pre-processing" Sep 10 20:09:06.298265 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298213 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" containerName="payload-pre-processing" Sep 10 20:09:06.298265 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298228 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" Sep 10 20:09:06.298265 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298234 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" Sep 10 20:09:06.298356 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298302 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a389f9e4-1490-4d5d-a52e-a6cbfd6e4970" containerName="payload-pre-processing" Sep 10 20:09:06.298356 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298309 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" Sep 10 20:09:06.298356 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.298319 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" Sep 10 20:09:06.300895 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.300878 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.301280 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.301256 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.301280 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.301281 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.301419 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.301290 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.301419 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.301299 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl42r\" (UniqueName: \"kubernetes.io/projected/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e-kube-api-access-gl42r\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:06.303386 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.303365 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-653653a4-dockercfg-4gknm\"" Sep 10 20:09:06.303483 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.303388 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-653653a4\"" Sep 10 20:09:06.315302 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.315277 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6"] Sep 10 20:09:06.321348 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.321321 2570 generic.go:356] "Generic (PLEG): container finished" podID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerID="6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199" exitCode=0 Sep 10 20:09:06.321444 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.321379 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" Sep 10 20:09:06.321444 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.321393 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" event={"ID":"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e","Type":"ContainerDied","Data":"6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199"} Sep 10 20:09:06.321529 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.321448 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5" event={"ID":"ce3a3a3e-a5a3-45f3-a210-443b64fddd2e","Type":"ContainerDied","Data":"db4996f42d95dd450b487c8de97617532abf77b316f5f7715bcc9a709b0fd215"} Sep 10 20:09:06.321529 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.321473 2570 scope.go:117] "RemoveContainer" containerID="6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199" Sep 10 20:09:06.332443 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.332409 2570 scope.go:117] "RemoveContainer" containerID="71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d" Sep 10 20:09:06.348931 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.348906 2570 scope.go:117] "RemoveContainer" containerID="6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199" Sep 10 20:09:06.349267 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:09:06.349244 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199\": container with ID starting with 6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199 not found: ID does not exist" containerID="6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199" Sep 10 20:09:06.349336 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.349275 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199"} err="failed to get container status \"6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199\": rpc error: code = NotFound desc = could not find container \"6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199\": container with ID starting with 6088a330e7763b22857b40c6df4348b28a244ccb6b616694cab26ef7d0519199 not found: ID does not exist" Sep 10 20:09:06.349336 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.349291 2570 scope.go:117] "RemoveContainer" containerID="71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d" Sep 10 20:09:06.349589 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:09:06.349565 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d\": container with ID starting with 71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d not found: ID does not exist" containerID="71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d" Sep 10 20:09:06.349639 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.349598 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d"} err="failed to get container status \"71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d\": rpc error: code = NotFound desc = could not find container \"71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d\": container with ID starting with 71b68e4204e90d797f27dcb28bc1e2bbd6b27e1fa42186cdaa5938529984ba6d not found: ID does not exist" Sep 10 20:09:06.360215 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.360194 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5"] Sep 10 20:09:06.365125 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.365103 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-468cfd8a-openshift-default-5bc5b8c88d-fclw5"] Sep 10 20:09:06.402149 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.402118 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.402280 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.402164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgrc\" (UniqueName: \"kubernetes.io/projected/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-kube-api-access-crgrc\") pod \"payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.503343 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.503310 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.503506 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.503357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgrc\" (UniqueName: \"kubernetes.io/projected/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-kube-api-access-crgrc\") pod \"payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.504087 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.504049 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.513410 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.513389 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgrc\" (UniqueName: \"kubernetes.io/projected/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-kube-api-access-crgrc\") pod \"payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.610964 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.610923 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:06.889510 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:06.889484 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6"] Sep 10 20:09:06.891832 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:09:06.891799 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e43c3f_cd3c_4e97_8f1c_3cb01e6d9e30.slice/crio-bf60afbfc72ff726d9113a34eec2c28283e20a3e9403e5c2c9121326bd9d5677 WatchSource:0}: Error finding container bf60afbfc72ff726d9113a34eec2c28283e20a3e9403e5c2c9121326bd9d5677: Status 404 returned error can't find the container with id bf60afbfc72ff726d9113a34eec2c28283e20a3e9403e5c2c9121326bd9d5677 Sep 10 20:09:07.328615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:07.328576 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" event={"ID":"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30","Type":"ContainerStarted","Data":"ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278"} Sep 10 20:09:07.329048 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:07.328630 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" event={"ID":"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30","Type":"ContainerStarted","Data":"bf60afbfc72ff726d9113a34eec2c28283e20a3e9403e5c2c9121326bd9d5677"} Sep 10 20:09:07.329048 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:07.328673 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:07.352097 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:07.352025 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" podStartSLOduration=1.352004494 podStartE2EDuration="1.352004494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:09:07.350008407 +0000 UTC m=+916.070955529" watchObservedRunningTime="2026-09-10 20:09:07.352004494 +0000 UTC m=+916.072951612" Sep 10 20:09:07.766308 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:07.766279 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" path="/var/lib/kubelet/pods/ce3a3a3e-a5a3-45f3-a210-443b64fddd2e/volumes" Sep 10 20:09:08.308704 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:08.308670 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d"] Sep 10 20:09:08.308964 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:08.308937 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" containerID="cri-o://9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d" gracePeriod=30 Sep 10 20:09:13.570431 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.570372 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:09:13.674378 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674354 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istiod-ca-cert\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674392 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-data\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674422 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-envoy\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674444 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-certs\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674490 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v65n\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-kube-api-access-7v65n\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674543 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-podinfo\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674575 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-credential-socket\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674923 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674619 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-token\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674923 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674655 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-socket\") pod \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\" (UID: \"a51ee1d0-47ee-46e6-8b00-2ace61a236c4\") " Sep 10 20:09:13.674923 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674638 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:13.674923 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674774 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:13.674923 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674790 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:09:13.675205 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.674988 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:13.675205 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.675048 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.675205 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.675054 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:13.675205 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.675096 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.675205 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.675110 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.676781 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.676750 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-kube-api-access-7v65n" (OuterVolumeSpecName: "kube-api-access-7v65n") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "kube-api-access-7v65n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:09:13.676912 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.676864 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:09:13.677048 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.677029 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:09:13.677132 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.677085 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a51ee1d0-47ee-46e6-8b00-2ace61a236c4" (UID: "a51ee1d0-47ee-46e6-8b00-2ace61a236c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:09:13.775741 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.775712 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.775741 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.775743 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.775926 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.775755 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.775926 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.775768 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v65n\" (UniqueName: \"kubernetes.io/projected/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-kube-api-access-7v65n\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.775926 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.775780 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.775926 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.775792 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51ee1d0-47ee-46e6-8b00-2ace61a236c4-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:13.935859 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.935646 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6"] Sep 10 20:09:13.936289 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.936266 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" Sep 10 20:09:13.936289 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.936292 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" Sep 10 20:09:13.936457 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.936331 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" Sep 10 20:09:13.936457 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.936342 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3a3a3e-a5a3-45f3-a210-443b64fddd2e" containerName="istio-proxy" Sep 10 20:09:13.936457 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.936443 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" Sep 10 20:09:13.936593 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.936459 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" Sep 10 20:09:13.939551 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.939528 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:13.942047 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.942027 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-589acd-dockercfg-hcgn4\"" Sep 10 20:09:13.942456 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.942436 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-589acd\"" Sep 10 20:09:13.951461 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.951414 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6"] Sep 10 20:09:13.978139 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.978108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjfpz\" (UniqueName: \"kubernetes.io/projected/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-kube-api-access-xjfpz\") pod \"payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:13.978339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:13.978317 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-plugins-config-volume\") pod \"payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:14.001322 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.001293 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh"] Sep 10 20:09:14.001726 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.001714 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" Sep 10 20:09:14.001782 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.001728 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerName="istio-proxy" Sep 10 20:09:14.004155 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.004140 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.020225 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.020201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh"] Sep 10 20:09:14.079569 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.079538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629fd5f2-7334-4075-855a-1ac1fd8c5f19-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.079736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.079606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-plugins-config-volume\") pod \"payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:14.079736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.079689 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbns2\" (UniqueName: \"kubernetes.io/projected/629fd5f2-7334-4075-855a-1ac1fd8c5f19-kube-api-access-rbns2\") pod \"payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.079736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.079731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjfpz\" (UniqueName: \"kubernetes.io/projected/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-kube-api-access-xjfpz\") pod \"payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:14.080276 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.080258 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-plugins-config-volume\") pod \"payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:14.088557 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.088531 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjfpz\" (UniqueName: \"kubernetes.io/projected/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-kube-api-access-xjfpz\") pod \"payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:14.181153 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.181115 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbns2\" (UniqueName: \"kubernetes.io/projected/629fd5f2-7334-4075-855a-1ac1fd8c5f19-kube-api-access-rbns2\") pod \"payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.181327 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.181199 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629fd5f2-7334-4075-855a-1ac1fd8c5f19-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.181904 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.181885 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629fd5f2-7334-4075-855a-1ac1fd8c5f19-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.190264 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.190188 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbns2\" (UniqueName: \"kubernetes.io/projected/629fd5f2-7334-4075-855a-1ac1fd8c5f19-kube-api-access-rbns2\") pod \"payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.252880 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.252849 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:14.313951 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.313859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:14.372114 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.372004 2570 generic.go:356] "Generic (PLEG): container finished" podID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" containerID="9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d" exitCode=0 Sep 10 20:09:14.372114 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.372063 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" Sep 10 20:09:14.372114 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.372063 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" event={"ID":"a51ee1d0-47ee-46e6-8b00-2ace61a236c4","Type":"ContainerDied","Data":"9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d"} Sep 10 20:09:14.372423 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.372147 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d" event={"ID":"a51ee1d0-47ee-46e6-8b00-2ace61a236c4","Type":"ContainerDied","Data":"f686c3415cdc1b43948e010b3669152ad3902c6754dc3a7443a76b7ff43a0c1b"} Sep 10 20:09:14.372423 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.372179 2570 scope.go:117] "RemoveContainer" containerID="9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d" Sep 10 20:09:14.387970 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.387886 2570 scope.go:117] "RemoveContainer" containerID="df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6" Sep 10 20:09:14.398245 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.398216 2570 scope.go:117] "RemoveContainer" containerID="9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d" Sep 10 20:09:14.398371 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.398318 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d"] Sep 10 20:09:14.398556 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:09:14.398534 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d\": container with ID starting with 9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d not found: ID does not exist" containerID="9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d" Sep 10 20:09:14.398620 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.398567 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d"} err="failed to get container status \"9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d\": rpc error: code = NotFound desc = could not find container \"9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d\": container with ID starting with 9fa94f26148b55ae04af4476c25dc2cf0082407db20c13158ded132564af342d not found: ID does not exist" Sep 10 20:09:14.398620 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.398590 2570 scope.go:117] "RemoveContainer" containerID="df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6" Sep 10 20:09:14.398846 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:09:14.398828 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6\": container with ID starting with df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6 not found: ID does not exist" containerID="df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6" Sep 10 20:09:14.398902 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.398849 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6"} err="failed to get container status \"df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6\": rpc error: code = NotFound desc = could not find container \"df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6\": container with ID starting with df6b38610a38b6e70b16a656df27c63dcb14bbb51db5a1bb98b9f2df7ff55ea6 not found: ID does not exist" Sep 10 20:09:14.401644 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.401621 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-28a5f0-openshift-default-5d9b9d4f4b-qjt4d"] Sep 10 20:09:14.542336 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.542310 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6"] Sep 10 20:09:14.544688 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:09:14.544654 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9d2bfa_0724_4906_ba95_9a1dd2b64431.slice/crio-981a2164d1506efa97d2477c5960f134b70147485ccb8fc60fcc2fd23be18899 WatchSource:0}: Error finding container 981a2164d1506efa97d2477c5960f134b70147485ccb8fc60fcc2fd23be18899: Status 404 returned error can't find the container with id 981a2164d1506efa97d2477c5960f134b70147485ccb8fc60fcc2fd23be18899 Sep 10 20:09:14.603851 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:14.603829 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh"] Sep 10 20:09:14.606558 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:09:14.606526 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629fd5f2_7334_4075_855a_1ac1fd8c5f19.slice/crio-d90e97532c03264f3a9c5fb82960717e3b88268a6f752241ce167288d46b1628 WatchSource:0}: Error finding container d90e97532c03264f3a9c5fb82960717e3b88268a6f752241ce167288d46b1628: Status 404 returned error can't find the container with id d90e97532c03264f3a9c5fb82960717e3b88268a6f752241ce167288d46b1628 Sep 10 20:09:15.379979 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.379938 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" event={"ID":"cc9d2bfa-0724-4906-ba95-9a1dd2b64431","Type":"ContainerStarted","Data":"829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e"} Sep 10 20:09:15.380203 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.380036 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" event={"ID":"cc9d2bfa-0724-4906-ba95-9a1dd2b64431","Type":"ContainerStarted","Data":"981a2164d1506efa97d2477c5960f134b70147485ccb8fc60fcc2fd23be18899"} Sep 10 20:09:15.380203 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.380084 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:15.381710 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.381684 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" event={"ID":"629fd5f2-7334-4075-855a-1ac1fd8c5f19","Type":"ContainerStarted","Data":"72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e"} Sep 10 20:09:15.381816 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.381725 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" event={"ID":"629fd5f2-7334-4075-855a-1ac1fd8c5f19","Type":"ContainerStarted","Data":"d90e97532c03264f3a9c5fb82960717e3b88268a6f752241ce167288d46b1628"} Sep 10 20:09:15.381872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.381833 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:15.401239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.401183 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" podStartSLOduration=2.40116534 podStartE2EDuration="2.40116534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:09:15.398778955 +0000 UTC m=+924.119726073" watchObservedRunningTime="2026-09-10 20:09:15.40116534 +0000 UTC m=+924.122112460" Sep 10 20:09:15.768365 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:15.768327 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a51ee1d0-47ee-46e6-8b00-2ace61a236c4" path="/var/lib/kubelet/pods/a51ee1d0-47ee-46e6-8b00-2ace61a236c4/volumes" Sep 10 20:09:18.337633 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:18.337599 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:18.355913 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:18.355865 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" podStartSLOduration=5.35584987 podStartE2EDuration="5.35584987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:09:15.422371988 +0000 UTC m=+924.143319108" watchObservedRunningTime="2026-09-10 20:09:18.35584987 +0000 UTC m=+927.076796989" Sep 10 20:09:26.389255 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:26.389224 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:09:26.389681 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:26.389465 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:09:29.280735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.280693 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4"] Sep 10 20:09:29.284636 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.284614 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:29.287426 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.287405 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:09:29.287592 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.287572 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:09:29.336297 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.336262 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4"] Sep 10 20:09:29.439492 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.439456 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-653653a4-xhsx4\" (UID: \"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:29.540443 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.540340 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-653653a4-xhsx4\" (UID: \"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:29.541006 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.540982 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-653653a4-xhsx4\" (UID: \"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:29.595416 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.595383 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:29.877879 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:29.877850 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4"] Sep 10 20:09:29.879807 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:09:29.879773 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f22f248_3d9c_4213_97b7_d48a4ad0b2e5.slice/crio-e06634f3c1e92de07b198aeeb06e9a9593fec60a02566b0abfc1fd083a0518be WatchSource:0}: Error finding container e06634f3c1e92de07b198aeeb06e9a9593fec60a02566b0abfc1fd083a0518be: Status 404 returned error can't find the container with id e06634f3c1e92de07b198aeeb06e9a9593fec60a02566b0abfc1fd083a0518be Sep 10 20:09:30.475000 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:30.474915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" event={"ID":"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5","Type":"ContainerStarted","Data":"b9f2a2092206b775f7a4bb1914dfbd45ee244aa1beb813e662efa2dad597a69c"} Sep 10 20:09:30.475000 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:30.474970 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" event={"ID":"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5","Type":"ContainerStarted","Data":"e06634f3c1e92de07b198aeeb06e9a9593fec60a02566b0abfc1fd083a0518be"} Sep 10 20:09:30.500321 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:30.500276 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" podStartSLOduration=1.500261054 podStartE2EDuration="1.500261054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:09:30.499343078 +0000 UTC m=+939.220290197" watchObservedRunningTime="2026-09-10 20:09:30.500261054 +0000 UTC m=+939.221208173" Sep 10 20:09:31.483168 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:31.483135 2570 generic.go:356] "Generic (PLEG): container finished" podID="6f22f248-3d9c-4213-97b7-d48a4ad0b2e5" containerID="b9f2a2092206b775f7a4bb1914dfbd45ee244aa1beb813e662efa2dad597a69c" exitCode=0 Sep 10 20:09:31.483540 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:31.483217 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" event={"ID":"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5","Type":"ContainerDied","Data":"b9f2a2092206b775f7a4bb1914dfbd45ee244aa1beb813e662efa2dad597a69c"} Sep 10 20:09:32.629063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:32.629039 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:32.772929 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:32.772843 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5-maas-api-service-ca\") pod \"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5\" (UID: \"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5\") " Sep 10 20:09:32.773272 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:32.773248 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f22f248-3d9c-4213-97b7-d48a4ad0b2e5" (UID: "6f22f248-3d9c-4213-97b7-d48a4ad0b2e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:09:32.873994 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:32.873961 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:33.502854 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:33.502822 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" Sep 10 20:09:33.502854 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:33.502846 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4" event={"ID":"6f22f248-3d9c-4213-97b7-d48a4ad0b2e5","Type":"ContainerDied","Data":"e06634f3c1e92de07b198aeeb06e9a9593fec60a02566b0abfc1fd083a0518be"} Sep 10 20:09:33.503101 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:33.502891 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e06634f3c1e92de07b198aeeb06e9a9593fec60a02566b0abfc1fd083a0518be" Sep 10 20:09:35.786039 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:35.786011 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6"] Sep 10 20:09:35.786865 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:35.786207 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" podUID="82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" containerName="payload-pre-processing" containerID="cri-o://ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278" gracePeriod=30 Sep 10 20:09:36.059796 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.059769 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:36.206219 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.206178 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4"] Sep 10 20:09:36.210231 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.210201 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-plugins-config-volume\") pod \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " Sep 10 20:09:36.210385 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.210236 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crgrc\" (UniqueName: \"kubernetes.io/projected/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-kube-api-access-crgrc\") pod \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\" (UID: \"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30\") " Sep 10 20:09:36.210582 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.210548 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" (UID: "82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:09:36.210706 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.210688 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:36.212593 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.212571 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-kube-api-access-crgrc" (OuterVolumeSpecName: "kube-api-access-crgrc") pod "82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" (UID: "82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30"). InnerVolumeSpecName "kube-api-access-crgrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:09:36.213415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.213396 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-653653a4-xhsx4"] Sep 10 20:09:36.311525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.311443 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crgrc\" (UniqueName: \"kubernetes.io/projected/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30-kube-api-access-crgrc\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:09:36.520456 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.520422 2570 generic.go:356] "Generic (PLEG): container finished" podID="82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" containerID="ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278" exitCode=0 Sep 10 20:09:36.520632 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.520471 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" Sep 10 20:09:36.520632 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.520487 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" event={"ID":"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30","Type":"ContainerDied","Data":"ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278"} Sep 10 20:09:36.520632 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.520532 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6" event={"ID":"82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30","Type":"ContainerDied","Data":"bf60afbfc72ff726d9113a34eec2c28283e20a3e9403e5c2c9121326bd9d5677"} Sep 10 20:09:36.520632 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.520553 2570 scope.go:117] "RemoveContainer" containerID="ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278" Sep 10 20:09:36.530782 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.530762 2570 scope.go:117] "RemoveContainer" containerID="ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278" Sep 10 20:09:36.531089 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:09:36.531056 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278\": container with ID starting with ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278 not found: ID does not exist" containerID="ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278" Sep 10 20:09:36.531169 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.531101 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278"} err="failed to get container status \"ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278\": rpc error: code = NotFound desc = could not find container \"ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278\": container with ID starting with ccb805fd7d4569398d52c6caf45d3f49ce7dd18d493a5cccbd6aef0188a5d278 not found: ID does not exist" Sep 10 20:09:36.553387 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.553363 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6"] Sep 10 20:09:36.567412 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:36.567353 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-653653a4-6fc6f85fb6-npbx6"] Sep 10 20:09:37.766016 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:37.765986 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f22f248-3d9c-4213-97b7-d48a4ad0b2e5" path="/var/lib/kubelet/pods/6f22f248-3d9c-4213-97b7-d48a4ad0b2e5/volumes" Sep 10 20:09:37.766442 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:09:37.766427 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" path="/var/lib/kubelet/pods/82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30/volumes" Sep 10 20:10:29.349633 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.349595 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6"] Sep 10 20:10:29.350210 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.349816 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" podUID="cc9d2bfa-0724-4906-ba95-9a1dd2b64431" containerName="payload-processing" containerID="cri-o://829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e" gracePeriod=30 Sep 10 20:10:29.388954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.388920 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh"] Sep 10 20:10:29.389277 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.389229 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" podUID="629fd5f2-7334-4075-855a-1ac1fd8c5f19" containerName="payload-pre-processing" containerID="cri-o://72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e" gracePeriod=30 Sep 10 20:10:29.695914 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.695887 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:10:29.698652 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.698634 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:10:29.809487 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.809399 2570 generic.go:356] "Generic (PLEG): container finished" podID="629fd5f2-7334-4075-855a-1ac1fd8c5f19" containerID="72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e" exitCode=0 Sep 10 20:10:29.809487 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.809443 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" Sep 10 20:10:29.809487 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.809458 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" event={"ID":"629fd5f2-7334-4075-855a-1ac1fd8c5f19","Type":"ContainerDied","Data":"72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e"} Sep 10 20:10:29.809778 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.809513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh" event={"ID":"629fd5f2-7334-4075-855a-1ac1fd8c5f19","Type":"ContainerDied","Data":"d90e97532c03264f3a9c5fb82960717e3b88268a6f752241ce167288d46b1628"} Sep 10 20:10:29.809778 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.809540 2570 scope.go:117] "RemoveContainer" containerID="72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e" Sep 10 20:10:29.810946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.810919 2570 generic.go:356] "Generic (PLEG): container finished" podID="cc9d2bfa-0724-4906-ba95-9a1dd2b64431" containerID="829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e" exitCode=0 Sep 10 20:10:29.811065 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.810958 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" Sep 10 20:10:29.811065 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.810962 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" event={"ID":"cc9d2bfa-0724-4906-ba95-9a1dd2b64431","Type":"ContainerDied","Data":"829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e"} Sep 10 20:10:29.811065 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.811003 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6" event={"ID":"cc9d2bfa-0724-4906-ba95-9a1dd2b64431","Type":"ContainerDied","Data":"981a2164d1506efa97d2477c5960f134b70147485ccb8fc60fcc2fd23be18899"} Sep 10 20:10:29.820128 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.820107 2570 scope.go:117] "RemoveContainer" containerID="72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e" Sep 10 20:10:29.820440 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:10:29.820411 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e\": container with ID starting with 72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e not found: ID does not exist" containerID="72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e" Sep 10 20:10:29.820552 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.820447 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e"} err="failed to get container status \"72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e\": rpc error: code = NotFound desc = could not find container \"72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e\": container with ID starting with 72387fb5c81394ff9692830f9382521a77e32721345816c340e042b0428a1c3e not found: ID does not exist" Sep 10 20:10:29.820552 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.820464 2570 scope.go:117] "RemoveContainer" containerID="829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e" Sep 10 20:10:29.830304 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.830230 2570 scope.go:117] "RemoveContainer" containerID="829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e" Sep 10 20:10:29.831810 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:10:29.831782 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e\": container with ID starting with 829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e not found: ID does not exist" containerID="829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e" Sep 10 20:10:29.831903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.831818 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e"} err="failed to get container status \"829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e\": rpc error: code = NotFound desc = could not find container \"829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e\": container with ID starting with 829d480c40b4ab778d5db0df0ef2c1b941ccef43f4f5d2e2dbd2e074a5e9f11e not found: ID does not exist" Sep 10 20:10:29.846208 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846184 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629fd5f2-7334-4075-855a-1ac1fd8c5f19-plugins-config-volume\") pod \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " Sep 10 20:10:29.846321 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846216 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbns2\" (UniqueName: \"kubernetes.io/projected/629fd5f2-7334-4075-855a-1ac1fd8c5f19-kube-api-access-rbns2\") pod \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\" (UID: \"629fd5f2-7334-4075-855a-1ac1fd8c5f19\") " Sep 10 20:10:29.846321 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846311 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-plugins-config-volume\") pod \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " Sep 10 20:10:29.846434 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846346 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjfpz\" (UniqueName: \"kubernetes.io/projected/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-kube-api-access-xjfpz\") pod \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\" (UID: \"cc9d2bfa-0724-4906-ba95-9a1dd2b64431\") " Sep 10 20:10:29.846617 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846595 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/629fd5f2-7334-4075-855a-1ac1fd8c5f19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "629fd5f2-7334-4075-855a-1ac1fd8c5f19" (UID: "629fd5f2-7334-4075-855a-1ac1fd8c5f19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:10:29.846698 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846677 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc9d2bfa-0724-4906-ba95-9a1dd2b64431" (UID: "cc9d2bfa-0724-4906-ba95-9a1dd2b64431"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:10:29.846748 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.846702 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629fd5f2-7334-4075-855a-1ac1fd8c5f19-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:10:29.848450 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.848429 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629fd5f2-7334-4075-855a-1ac1fd8c5f19-kube-api-access-rbns2" (OuterVolumeSpecName: "kube-api-access-rbns2") pod "629fd5f2-7334-4075-855a-1ac1fd8c5f19" (UID: "629fd5f2-7334-4075-855a-1ac1fd8c5f19"). InnerVolumeSpecName "kube-api-access-rbns2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:10:29.848518 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.848444 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-kube-api-access-xjfpz" (OuterVolumeSpecName: "kube-api-access-xjfpz") pod "cc9d2bfa-0724-4906-ba95-9a1dd2b64431" (UID: "cc9d2bfa-0724-4906-ba95-9a1dd2b64431"). InnerVolumeSpecName "kube-api-access-xjfpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:10:29.948194 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.948157 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbns2\" (UniqueName: \"kubernetes.io/projected/629fd5f2-7334-4075-855a-1ac1fd8c5f19-kube-api-access-rbns2\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:10:29.948194 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.948187 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:10:29.948194 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:29.948197 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjfpz\" (UniqueName: \"kubernetes.io/projected/cc9d2bfa-0724-4906-ba95-9a1dd2b64431-kube-api-access-xjfpz\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:10:30.143183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:30.143155 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6"] Sep 10 20:10:30.152532 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:30.152507 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-589acd-dcd9494b5-h2fn6"] Sep 10 20:10:30.171659 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:30.171630 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh"] Sep 10 20:10:30.179553 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:30.179527 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-589acd-d9b6dc58b-6wqmh"] Sep 10 20:10:31.765715 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:31.765681 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="629fd5f2-7334-4075-855a-1ac1fd8c5f19" path="/var/lib/kubelet/pods/629fd5f2-7334-4075-855a-1ac1fd8c5f19/volumes" Sep 10 20:10:31.766120 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:31.766030 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc9d2bfa-0724-4906-ba95-9a1dd2b64431" path="/var/lib/kubelet/pods/cc9d2bfa-0724-4906-ba95-9a1dd2b64431/volumes" Sep 10 20:10:41.196561 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.196525 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b"] Sep 10 20:10:41.197238 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197216 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc9d2bfa-0724-4906-ba95-9a1dd2b64431" containerName="payload-processing" Sep 10 20:10:41.197238 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197238 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc9d2bfa-0724-4906-ba95-9a1dd2b64431" containerName="payload-processing" Sep 10 20:10:41.197394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197251 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" containerName="payload-pre-processing" Sep 10 20:10:41.197394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197261 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" containerName="payload-pre-processing" Sep 10 20:10:41.197394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197269 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="629fd5f2-7334-4075-855a-1ac1fd8c5f19" containerName="payload-pre-processing" Sep 10 20:10:41.197394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197277 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="629fd5f2-7334-4075-855a-1ac1fd8c5f19" containerName="payload-pre-processing" Sep 10 20:10:41.197394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197311 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f22f248-3d9c-4213-97b7-d48a4ad0b2e5" containerName="revoke-keys" Sep 10 20:10:41.197394 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197318 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f22f248-3d9c-4213-97b7-d48a4ad0b2e5" containerName="revoke-keys" Sep 10 20:10:41.197703 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197407 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="629fd5f2-7334-4075-855a-1ac1fd8c5f19" containerName="payload-pre-processing" Sep 10 20:10:41.197703 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197418 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f22f248-3d9c-4213-97b7-d48a4ad0b2e5" containerName="revoke-keys" Sep 10 20:10:41.197703 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197430 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="82e43c3f-cd3c-4e97-8f1c-3cb01e6d9e30" containerName="payload-pre-processing" Sep 10 20:10:41.197703 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.197441 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc9d2bfa-0724-4906-ba95-9a1dd2b64431" containerName="payload-processing" Sep 10 20:10:41.202445 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.202423 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.208641 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.208618 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-842160-openshift-default-dockercfg-8lnj8\"" Sep 10 20:10:41.225713 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.225685 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b"] Sep 10 20:10:41.365339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd7617e-013b-4843-a114-ca81996956ad-istio-podinfo\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365330 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd7617e-013b-4843-a114-ca81996956ad-istiod-ca-cert\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365424 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-credential-socket\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365467 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-envoy\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365496 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-certs\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365513 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-istio-token\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-socket\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmdz8\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-kube-api-access-zmdz8\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.365614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.365607 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-data\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466426 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-envoy\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466494 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-certs\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466525 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-istio-token\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466811 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466554 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-socket\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466811 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466614 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmdz8\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-kube-api-access-zmdz8\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466811 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-data\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.466981 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466945 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-socket\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.467157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.466815 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd7617e-013b-4843-a114-ca81996956ad-istio-podinfo\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.467384 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.467357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd7617e-013b-4843-a114-ca81996956ad-istiod-ca-cert\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.467483 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.467445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-credential-socket\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.467546 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.467233 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-certs\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.467546 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.467185 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-data\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.467876 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.467850 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-credential-socket\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.468236 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.468210 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd7617e-013b-4843-a114-ca81996956ad-istiod-ca-cert\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.469246 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.469217 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-envoy\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.470025 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.470000 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd7617e-013b-4843-a114-ca81996956ad-istio-podinfo\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.486831 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.486810 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmdz8\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-kube-api-access-zmdz8\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.488881 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.488864 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-istio-token\") pod \"e2e-noten-842160-openshift-default-54df884cd-hqd2b\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.514934 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.514905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:41.809352 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.809309 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b"] Sep 10 20:10:41.812959 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:10:41.812935 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd7617e_013b_4843_a114_ca81996956ad.slice/crio-a1be6120c317291be9abd0035aba8fd8f797d32e911da2bdfdd4df5e4b211f8f WatchSource:0}: Error finding container a1be6120c317291be9abd0035aba8fd8f797d32e911da2bdfdd4df5e4b211f8f: Status 404 returned error can't find the container with id a1be6120c317291be9abd0035aba8fd8f797d32e911da2bdfdd4df5e4b211f8f Sep 10 20:10:41.815081 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.815039 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:10:41.815144 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.815117 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:10:41.815181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.815152 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:10:41.881482 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.881457 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" event={"ID":"0dd7617e-013b-4843-a114-ca81996956ad","Type":"ContainerStarted","Data":"4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8"} Sep 10 20:10:41.881574 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.881505 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" event={"ID":"0dd7617e-013b-4843-a114-ca81996956ad","Type":"ContainerStarted","Data":"a1be6120c317291be9abd0035aba8fd8f797d32e911da2bdfdd4df5e4b211f8f"} Sep 10 20:10:41.921542 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:41.921481 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" podStartSLOduration=0.921464788 podStartE2EDuration="921.464788ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:10:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:10:41.91983626 +0000 UTC m=+1010.640783380" watchObservedRunningTime="2026-09-10 20:10:41.921464788 +0000 UTC m=+1010.642411907" Sep 10 20:10:42.515700 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:42.515665 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:42.520461 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:42.520434 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:42.887183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:42.887094 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:42.887992 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:42.887970 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:10:48.050744 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.050708 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9"] Sep 10 20:10:48.051208 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.050948 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" podUID="df003b36-363e-4a79-b7da-07c9a94f6405" containerName="payload-processing" containerID="cri-o://e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9" gracePeriod=30 Sep 10 20:10:48.303700 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.303637 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:10:48.339855 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.339823 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df003b36-363e-4a79-b7da-07c9a94f6405-plugins-config-volume\") pod \"df003b36-363e-4a79-b7da-07c9a94f6405\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " Sep 10 20:10:48.340042 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.339881 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmwdx\" (UniqueName: \"kubernetes.io/projected/df003b36-363e-4a79-b7da-07c9a94f6405-kube-api-access-mmwdx\") pod \"df003b36-363e-4a79-b7da-07c9a94f6405\" (UID: \"df003b36-363e-4a79-b7da-07c9a94f6405\") " Sep 10 20:10:48.340174 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.340150 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df003b36-363e-4a79-b7da-07c9a94f6405-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df003b36-363e-4a79-b7da-07c9a94f6405" (UID: "df003b36-363e-4a79-b7da-07c9a94f6405"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:10:48.342146 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.342116 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df003b36-363e-4a79-b7da-07c9a94f6405-kube-api-access-mmwdx" (OuterVolumeSpecName: "kube-api-access-mmwdx") pod "df003b36-363e-4a79-b7da-07c9a94f6405" (UID: "df003b36-363e-4a79-b7da-07c9a94f6405"). InnerVolumeSpecName "kube-api-access-mmwdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:10:48.441087 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.441051 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df003b36-363e-4a79-b7da-07c9a94f6405-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:10:48.441262 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.441105 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmwdx\" (UniqueName: \"kubernetes.io/projected/df003b36-363e-4a79-b7da-07c9a94f6405-kube-api-access-mmwdx\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:10:48.919598 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.919563 2570 generic.go:356] "Generic (PLEG): container finished" podID="df003b36-363e-4a79-b7da-07c9a94f6405" containerID="e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9" exitCode=0 Sep 10 20:10:48.919756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.919611 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" Sep 10 20:10:48.919756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.919652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" event={"ID":"df003b36-363e-4a79-b7da-07c9a94f6405","Type":"ContainerDied","Data":"e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9"} Sep 10 20:10:48.919756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.919710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9" event={"ID":"df003b36-363e-4a79-b7da-07c9a94f6405","Type":"ContainerDied","Data":"3124b84d5ad36492cb188def2f485f04bc9074f201a94e982ad25f8c23c671d8"} Sep 10 20:10:48.919756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.919737 2570 scope.go:117] "RemoveContainer" containerID="e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9" Sep 10 20:10:48.929060 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.929043 2570 scope.go:117] "RemoveContainer" containerID="e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9" Sep 10 20:10:48.929376 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:10:48.929355 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9\": container with ID starting with e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9 not found: ID does not exist" containerID="e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9" Sep 10 20:10:48.929435 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.929384 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9"} err="failed to get container status \"e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9\": rpc error: code = NotFound desc = could not find container \"e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9\": container with ID starting with e79c6021c628b283959c5df1ca7d6c524b5066b7560bbf61d2ba41c0271411c9 not found: ID does not exist" Sep 10 20:10:48.974415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.974387 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9"] Sep 10 20:10:48.982441 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:48.982420 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6d8d62-f95c6f6f-7dqp9"] Sep 10 20:10:49.773100 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:49.773051 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df003b36-363e-4a79-b7da-07c9a94f6405" path="/var/lib/kubelet/pods/df003b36-363e-4a79-b7da-07c9a94f6405/volumes" Sep 10 20:10:57.624111 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:57.624051 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz"] Sep 10 20:10:57.624574 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:57.624344 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" containerID="cri-o://3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787" gracePeriod=30 Sep 10 20:10:59.338468 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:59.338428 2570 patch_prober.go:28] interesting pod/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 20:10:59.338903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:10:59.338525 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 20:11:02.887645 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.887581 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:11:02.990389 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990358 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8d764-30b0-4afa-a9fc-4d192754e51f-istiod-ca-cert\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990389 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990399 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vrfw\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-kube-api-access-9vrfw\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990652 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990429 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-credential-socket\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990652 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990462 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-certs\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990652 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990543 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-podinfo\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990652 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990609 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-envoy\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990652 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990637 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-token\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990694 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-data\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990728 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-socket\") pod \"73a8d764-30b0-4afa-a9fc-4d192754e51f\" (UID: \"73a8d764-30b0-4afa-a9fc-4d192754e51f\") " Sep 10 20:11:02.990917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990760 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:02.990917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990776 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:02.990917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.990908 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a8d764-30b0-4afa-a9fc-4d192754e51f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:11:02.991210 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991159 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:02.991210 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991177 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-data" (OuterVolumeSpecName: "istio-data") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:02.991317 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991298 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:02.991380 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991318 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:02.991380 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991332 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a8d764-30b0-4afa-a9fc-4d192754e51f-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:02.991380 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991346 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:02.991380 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.991358 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:02.993109 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.993064 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:02.993241 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.993186 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-token" (OuterVolumeSpecName: "istio-token") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:02.993241 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.993190 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-kube-api-access-9vrfw" (OuterVolumeSpecName: "kube-api-access-9vrfw") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "kube-api-access-9vrfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:02.993372 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.993274 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73a8d764-30b0-4afa-a9fc-4d192754e51f" (UID: "73a8d764-30b0-4afa-a9fc-4d192754e51f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:11:02.994065 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.994039 2570 generic.go:356] "Generic (PLEG): container finished" podID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerID="3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787" exitCode=0 Sep 10 20:11:02.994184 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.994107 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" Sep 10 20:11:02.994184 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.994129 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" event={"ID":"73a8d764-30b0-4afa-a9fc-4d192754e51f","Type":"ContainerDied","Data":"3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787"} Sep 10 20:11:02.994266 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.994185 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz" event={"ID":"73a8d764-30b0-4afa-a9fc-4d192754e51f","Type":"ContainerDied","Data":"13220af37546b94320005bec1ee6a74ebcd58ee55fdc7455167bdd1a3d978619"} Sep 10 20:11:02.994266 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:02.994216 2570 scope.go:117] "RemoveContainer" containerID="3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787" Sep 10 20:11:03.014376 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.014355 2570 scope.go:117] "RemoveContainer" containerID="bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3" Sep 10 20:11:03.022855 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.022834 2570 scope.go:117] "RemoveContainer" containerID="3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787" Sep 10 20:11:03.023199 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:11:03.023177 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787\": container with ID starting with 3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787 not found: ID does not exist" containerID="3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787" Sep 10 20:11:03.023259 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.023208 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787"} err="failed to get container status \"3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787\": rpc error: code = NotFound desc = could not find container \"3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787\": container with ID starting with 3e86e7bb197d50a3f565e6aa2b98f8284fc40f758d7214bdd52ec705ec015787 not found: ID does not exist" Sep 10 20:11:03.023259 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.023226 2570 scope.go:117] "RemoveContainer" containerID="bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3" Sep 10 20:11:03.023485 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:11:03.023467 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3\": container with ID starting with bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3 not found: ID does not exist" containerID="bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3" Sep 10 20:11:03.023529 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.023490 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3"} err="failed to get container status \"bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3\": rpc error: code = NotFound desc = could not find container \"bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3\": container with ID starting with bd71dd46a7bdfd380cfb234ee1508ad6e4b5f7a8b4249341a95fcbbf6693e2a3 not found: ID does not exist" Sep 10 20:11:03.037808 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.037780 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz"] Sep 10 20:11:03.049016 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.048986 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6d8d62-openshift-default-5f88bd6686-bfktz"] Sep 10 20:11:03.092732 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.092699 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vrfw\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-kube-api-access-9vrfw\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:03.092732 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.092726 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:03.092732 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.092739 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:03.092986 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.092751 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a8d764-30b0-4afa-a9fc-4d192754e51f-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:03.766723 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:03.766690 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" path="/var/lib/kubelet/pods/73a8d764-30b0-4afa-a9fc-4d192754e51f/volumes" Sep 10 20:11:08.855414 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:08.855372 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg"] Sep 10 20:11:08.855887 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:08.855629 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" podUID="1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" containerName="payload-pre-processing" containerID="cri-o://ac76307cb9cdd53bd773e3b1efab834e3788bcd5cbb23976ff5265dc21eb06e6" gracePeriod=30 Sep 10 20:11:09.029991 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.029962 2570 generic.go:356] "Generic (PLEG): container finished" podID="1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" containerID="ac76307cb9cdd53bd773e3b1efab834e3788bcd5cbb23976ff5265dc21eb06e6" exitCode=0 Sep 10 20:11:09.030156 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.030040 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" event={"ID":"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c","Type":"ContainerDied","Data":"ac76307cb9cdd53bd773e3b1efab834e3788bcd5cbb23976ff5265dc21eb06e6"} Sep 10 20:11:09.121427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.121400 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:11:09.249687 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.249649 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-plugins-config-volume\") pod \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " Sep 10 20:11:09.249892 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.249740 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ks8h\" (UniqueName: \"kubernetes.io/projected/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-kube-api-access-4ks8h\") pod \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\" (UID: \"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c\") " Sep 10 20:11:09.250133 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.250108 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" (UID: "1350ed22-49e9-4c1b-875e-2a54e3ca4f6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:11:09.251829 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.251805 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-kube-api-access-4ks8h" (OuterVolumeSpecName: "kube-api-access-4ks8h") pod "1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" (UID: "1350ed22-49e9-4c1b-875e-2a54e3ca4f6c"). InnerVolumeSpecName "kube-api-access-4ks8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:09.350669 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.350636 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:09.350669 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:09.350665 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ks8h\" (UniqueName: \"kubernetes.io/projected/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c-kube-api-access-4ks8h\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:10.037993 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:10.037898 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" event={"ID":"1350ed22-49e9-4c1b-875e-2a54e3ca4f6c","Type":"ContainerDied","Data":"300f916d95c0640224f758aeed2e5a5f6e6ed330598741974294f021d101c346"} Sep 10 20:11:10.037993 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:10.037925 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg" Sep 10 20:11:10.037993 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:10.037976 2570 scope.go:117] "RemoveContainer" containerID="ac76307cb9cdd53bd773e3b1efab834e3788bcd5cbb23976ff5265dc21eb06e6" Sep 10 20:11:10.059416 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:10.059391 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg"] Sep 10 20:11:10.062254 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:10.062232 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39487c-696cd6598-8l6dg"] Sep 10 20:11:11.765243 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:11.765207 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" path="/var/lib/kubelet/pods/1350ed22-49e9-4c1b-875e-2a54e3ca4f6c/volumes" Sep 10 20:11:15.913887 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:15.913847 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5"] Sep 10 20:11:15.914424 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:15.914109 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" containerID="cri-o://83d8cb25fe8014fab516625b70bfe2590f61cba9f760f1fa85f3ffb0de661b31" gracePeriod=30 Sep 10 20:11:21.099835 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.099798 2570 generic.go:356] "Generic (PLEG): container finished" podID="b1999064-b434-43fe-97fe-237be43d9b07" containerID="83d8cb25fe8014fab516625b70bfe2590f61cba9f760f1fa85f3ffb0de661b31" exitCode=0 Sep 10 20:11:21.100275 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.099868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" event={"ID":"b1999064-b434-43fe-97fe-237be43d9b07","Type":"ContainerDied","Data":"83d8cb25fe8014fab516625b70bfe2590f61cba9f760f1fa85f3ffb0de661b31"} Sep 10 20:11:21.100275 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.099961 2570 scope.go:117] "RemoveContainer" containerID="765b292e8330f9637c4e83e03ee254d7b8bf95e205671975fa611e03439def81" Sep 10 20:11:21.181913 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.181845 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:11:21.267159 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267123 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-data\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267159 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267167 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1999064-b434-43fe-97fe-237be43d9b07-istiod-ca-cert\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267186 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-envoy\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267227 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1999064-b434-43fe-97fe-237be43d9b07-istio-podinfo\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267249 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-istio-token\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267275 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-certs\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267295 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-credential-socket\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267329 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttfj4\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-kube-api-access-ttfj4\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267372 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-socket\") pod \"b1999064-b434-43fe-97fe-237be43d9b07\" (UID: \"b1999064-b434-43fe-97fe-237be43d9b07\") " Sep 10 20:11:21.267430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267398 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:21.267828 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267631 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:21.267828 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267646 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:21.267828 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267658 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1999064-b434-43fe-97fe-237be43d9b07-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:11:21.267828 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267667 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:21.268008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267834 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.268008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267856 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1999064-b434-43fe-97fe-237be43d9b07-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.268008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267872 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.268008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267886 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.268008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.267899 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.269686 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.269656 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:21.269686 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.269663 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-kube-api-access-ttfj4" (OuterVolumeSpecName: "kube-api-access-ttfj4") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "kube-api-access-ttfj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:21.269952 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.269933 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:21.270011 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.269973 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1999064-b434-43fe-97fe-237be43d9b07-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1999064-b434-43fe-97fe-237be43d9b07" (UID: "b1999064-b434-43fe-97fe-237be43d9b07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:11:21.369260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.369222 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1999064-b434-43fe-97fe-237be43d9b07-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.369260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.369250 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1999064-b434-43fe-97fe-237be43d9b07-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.369260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.369259 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.369260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.369268 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttfj4\" (UniqueName: \"kubernetes.io/projected/b1999064-b434-43fe-97fe-237be43d9b07-kube-api-access-ttfj4\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:21.941559 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941523 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq"] Sep 10 20:11:21.941943 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941930 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" Sep 10 20:11:21.941988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941947 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" Sep 10 20:11:21.941988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941962 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df003b36-363e-4a79-b7da-07c9a94f6405" containerName="payload-processing" Sep 10 20:11:21.941988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941968 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="df003b36-363e-4a79-b7da-07c9a94f6405" containerName="payload-processing" Sep 10 20:11:21.941988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941980 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" containerName="payload-pre-processing" Sep 10 20:11:21.941988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941987 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" containerName="payload-pre-processing" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.941994 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942000 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942010 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942015 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942098 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942107 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942114 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942121 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1350ed22-49e9-4c1b-875e-2a54e3ca4f6c" containerName="payload-pre-processing" Sep 10 20:11:21.942154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.942127 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="df003b36-363e-4a79-b7da-07c9a94f6405" containerName="payload-processing" Sep 10 20:11:21.945588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.945567 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:21.948231 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.948213 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-591d7c\"" Sep 10 20:11:21.961725 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.961701 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq"] Sep 10 20:11:21.974785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.974753 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv9l9\" (UniqueName: \"kubernetes.io/projected/d924d5ac-ac01-4879-8dfb-a4914e21e77a-kube-api-access-mv9l9\") pod \"payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:21.974930 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:21.974827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d924d5ac-ac01-4879-8dfb-a4914e21e77a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:22.075399 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.075310 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d924d5ac-ac01-4879-8dfb-a4914e21e77a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:22.075564 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.075418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv9l9\" (UniqueName: \"kubernetes.io/projected/d924d5ac-ac01-4879-8dfb-a4914e21e77a-kube-api-access-mv9l9\") pod \"payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:22.076279 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.076251 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d924d5ac-ac01-4879-8dfb-a4914e21e77a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:22.089840 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.089806 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc"] Sep 10 20:11:22.090297 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.090260 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv9l9\" (UniqueName: \"kubernetes.io/projected/d924d5ac-ac01-4879-8dfb-a4914e21e77a-kube-api-access-mv9l9\") pod \"payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:22.090462 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.090444 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" Sep 10 20:11:22.090529 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.090466 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1999064-b434-43fe-97fe-237be43d9b07" containerName="istio-proxy" Sep 10 20:11:22.090596 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.090585 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a8d764-30b0-4afa-a9fc-4d192754e51f" containerName="istio-proxy" Sep 10 20:11:22.099902 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.099877 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.104804 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.104204 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-591d7c-dockercfg-p49qt\"" Sep 10 20:11:22.107148 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.107064 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc"] Sep 10 20:11:22.110638 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.110607 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" event={"ID":"b1999064-b434-43fe-97fe-237be43d9b07","Type":"ContainerDied","Data":"2457fa65758da01355d96f5ff57cab6f29591e085a55e49321e425b2abe9eebc"} Sep 10 20:11:22.110775 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.110675 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5" Sep 10 20:11:22.110775 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.110693 2570 scope.go:117] "RemoveContainer" containerID="83d8cb25fe8014fab516625b70bfe2590f61cba9f760f1fa85f3ffb0de661b31" Sep 10 20:11:22.143181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.143024 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5"] Sep 10 20:11:22.149903 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.149875 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-39487c-openshift-default-849b79694d-n8ql5"] Sep 10 20:11:22.176390 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.176354 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587e9320-e77b-4907-ae02-4eae39ab565e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.176553 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.176402 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lrrj\" (UniqueName: \"kubernetes.io/projected/587e9320-e77b-4907-ae02-4eae39ab565e-kube-api-access-8lrrj\") pod \"payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.256382 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.256351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:22.277265 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.277235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587e9320-e77b-4907-ae02-4eae39ab565e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.277388 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.277296 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lrrj\" (UniqueName: \"kubernetes.io/projected/587e9320-e77b-4907-ae02-4eae39ab565e-kube-api-access-8lrrj\") pod \"payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.277943 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.277920 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587e9320-e77b-4907-ae02-4eae39ab565e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.289685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.289659 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lrrj\" (UniqueName: \"kubernetes.io/projected/587e9320-e77b-4907-ae02-4eae39ab565e-kube-api-access-8lrrj\") pod \"payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.416949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.416854 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:22.535854 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.535826 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq"] Sep 10 20:11:22.537249 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:11:22.537222 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd924d5ac_ac01_4879_8dfb_a4914e21e77a.slice/crio-d1ca4693e6228cb7ea569b2b76cc88af95d9db284bac317c1ee8726c08c70014 WatchSource:0}: Error finding container d1ca4693e6228cb7ea569b2b76cc88af95d9db284bac317c1ee8726c08c70014: Status 404 returned error can't find the container with id d1ca4693e6228cb7ea569b2b76cc88af95d9db284bac317c1ee8726c08c70014 Sep 10 20:11:22.693401 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:22.693376 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc"] Sep 10 20:11:22.695478 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:11:22.695455 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587e9320_e77b_4907_ae02_4eae39ab565e.slice/crio-868e1bb7d2fa6e18871278d48e5a3a1bad5b9ddc9ac5ae2bdc69e024a4037116 WatchSource:0}: Error finding container 868e1bb7d2fa6e18871278d48e5a3a1bad5b9ddc9ac5ae2bdc69e024a4037116: Status 404 returned error can't find the container with id 868e1bb7d2fa6e18871278d48e5a3a1bad5b9ddc9ac5ae2bdc69e024a4037116 Sep 10 20:11:23.117950 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.117915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" event={"ID":"587e9320-e77b-4907-ae02-4eae39ab565e","Type":"ContainerStarted","Data":"7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca"} Sep 10 20:11:23.118409 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.117970 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" event={"ID":"587e9320-e77b-4907-ae02-4eae39ab565e","Type":"ContainerStarted","Data":"868e1bb7d2fa6e18871278d48e5a3a1bad5b9ddc9ac5ae2bdc69e024a4037116"} Sep 10 20:11:23.118409 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.118109 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:23.120699 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.120667 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" event={"ID":"d924d5ac-ac01-4879-8dfb-a4914e21e77a","Type":"ContainerStarted","Data":"1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5"} Sep 10 20:11:23.120808 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.120707 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" event={"ID":"d924d5ac-ac01-4879-8dfb-a4914e21e77a","Type":"ContainerStarted","Data":"d1ca4693e6228cb7ea569b2b76cc88af95d9db284bac317c1ee8726c08c70014"} Sep 10 20:11:23.120808 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.120802 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:23.141420 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.141365 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" podStartSLOduration=1.141344776 podStartE2EDuration="1.141344776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:11:23.139775426 +0000 UTC m=+1051.860722544" watchObservedRunningTime="2026-09-10 20:11:23.141344776 +0000 UTC m=+1051.862291899" Sep 10 20:11:23.161961 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.161908 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" podStartSLOduration=2.161889037 podStartE2EDuration="2.161889037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:11:23.159861187 +0000 UTC m=+1051.880808307" watchObservedRunningTime="2026-09-10 20:11:23.161889037 +0000 UTC m=+1051.882836155" Sep 10 20:11:23.766992 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:23.766958 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1999064-b434-43fe-97fe-237be43d9b07" path="/var/lib/kubelet/pods/b1999064-b434-43fe-97fe-237be43d9b07/volumes" Sep 10 20:11:26.127769 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.127733 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq"] Sep 10 20:11:26.131633 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.131612 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.135502 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.135477 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2017ae11\"" Sep 10 20:11:26.135985 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.135963 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2017ae11-dockercfg-l6xls\"" Sep 10 20:11:26.153644 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.153612 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq"] Sep 10 20:11:26.212591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.212553 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8spt4\" (UniqueName: \"kubernetes.io/projected/95797dab-0df2-4dd4-b0f9-608e27a02ed3-kube-api-access-8spt4\") pod \"payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.212591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.212593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95797dab-0df2-4dd4-b0f9-608e27a02ed3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.244382 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.244332 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k"] Sep 10 20:11:26.248638 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.248615 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.288490 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.288455 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k"] Sep 10 20:11:26.314088 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.314039 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ce090c0-6658-47e0-84d0-e3c171ef23da-plugins-config-volume\") pod \"payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.314272 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.314159 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f9p7\" (UniqueName: \"kubernetes.io/projected/4ce090c0-6658-47e0-84d0-e3c171ef23da-kube-api-access-8f9p7\") pod \"payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.314272 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.314239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8spt4\" (UniqueName: \"kubernetes.io/projected/95797dab-0df2-4dd4-b0f9-608e27a02ed3-kube-api-access-8spt4\") pod \"payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.314384 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.314275 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95797dab-0df2-4dd4-b0f9-608e27a02ed3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.314961 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.314942 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95797dab-0df2-4dd4-b0f9-608e27a02ed3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.334416 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.334381 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8spt4\" (UniqueName: \"kubernetes.io/projected/95797dab-0df2-4dd4-b0f9-608e27a02ed3-kube-api-access-8spt4\") pod \"payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.414945 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.414849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ce090c0-6658-47e0-84d0-e3c171ef23da-plugins-config-volume\") pod \"payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.414945 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.414933 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f9p7\" (UniqueName: \"kubernetes.io/projected/4ce090c0-6658-47e0-84d0-e3c171ef23da-kube-api-access-8f9p7\") pod \"payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.415593 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.415575 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ce090c0-6658-47e0-84d0-e3c171ef23da-plugins-config-volume\") pod \"payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.427707 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.427679 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f9p7\" (UniqueName: \"kubernetes.io/projected/4ce090c0-6658-47e0-84d0-e3c171ef23da-kube-api-access-8f9p7\") pod \"payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.444443 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.444411 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:26.558785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.558749 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:26.727979 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.727954 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq"] Sep 10 20:11:26.729807 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:11:26.729782 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95797dab_0df2_4dd4_b0f9_608e27a02ed3.slice/crio-b8f9606649ef9722e078fdd7d4999819eeb6696496a4bee1631f733cba312b35 WatchSource:0}: Error finding container b8f9606649ef9722e078fdd7d4999819eeb6696496a4bee1631f733cba312b35: Status 404 returned error can't find the container with id b8f9606649ef9722e078fdd7d4999819eeb6696496a4bee1631f733cba312b35 Sep 10 20:11:26.839869 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:26.839835 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k"] Sep 10 20:11:26.840864 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:11:26.840820 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce090c0_6658_47e0_84d0_e3c171ef23da.slice/crio-c9c7275d9aa9ea0c1e9bf9239865248ed9fdfaf9543b793cffc0e280984529d1 WatchSource:0}: Error finding container c9c7275d9aa9ea0c1e9bf9239865248ed9fdfaf9543b793cffc0e280984529d1: Status 404 returned error can't find the container with id c9c7275d9aa9ea0c1e9bf9239865248ed9fdfaf9543b793cffc0e280984529d1 Sep 10 20:11:27.145553 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.145501 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" event={"ID":"95797dab-0df2-4dd4-b0f9-608e27a02ed3","Type":"ContainerStarted","Data":"5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65"} Sep 10 20:11:27.146149 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.146122 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:27.146291 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.146162 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" event={"ID":"95797dab-0df2-4dd4-b0f9-608e27a02ed3","Type":"ContainerStarted","Data":"b8f9606649ef9722e078fdd7d4999819eeb6696496a4bee1631f733cba312b35"} Sep 10 20:11:27.147856 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.147824 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" event={"ID":"4ce090c0-6658-47e0-84d0-e3c171ef23da","Type":"ContainerStarted","Data":"be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc"} Sep 10 20:11:27.147970 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.147867 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" event={"ID":"4ce090c0-6658-47e0-84d0-e3c171ef23da","Type":"ContainerStarted","Data":"c9c7275d9aa9ea0c1e9bf9239865248ed9fdfaf9543b793cffc0e280984529d1"} Sep 10 20:11:27.147970 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.147896 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:27.173909 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.173859 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" podStartSLOduration=1.173842086 podStartE2EDuration="1.173842086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:11:27.172145781 +0000 UTC m=+1055.893092900" watchObservedRunningTime="2026-09-10 20:11:27.173842086 +0000 UTC m=+1055.894789205" Sep 10 20:11:27.194950 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:27.194886 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" podStartSLOduration=1.194868397 podStartE2EDuration="1.194868397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:11:27.19419664 +0000 UTC m=+1055.915143760" watchObservedRunningTime="2026-09-10 20:11:27.194868397 +0000 UTC m=+1055.915815516" Sep 10 20:11:34.128268 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:34.128236 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:11:34.128693 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:34.128309 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:11:34.305315 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:34.305275 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b"] Sep 10 20:11:34.305669 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:34.305619 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" podUID="0dd7617e-013b-4843-a114-ca81996956ad" containerName="istio-proxy" containerID="cri-o://4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8" gracePeriod=30 Sep 10 20:11:38.155611 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:38.155573 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:11:38.155992 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:38.155728 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:11:39.562392 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.562364 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:11:39.648331 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648246 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-socket\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648331 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648300 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-data\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648346 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-envoy\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648388 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd7617e-013b-4843-a114-ca81996956ad-istio-podinfo\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648418 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd7617e-013b-4843-a114-ca81996956ad-istiod-ca-cert\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648441 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-credential-socket\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648466 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmdz8\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-kube-api-access-zmdz8\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648530 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-certs\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648551 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-istio-token\") pod \"0dd7617e-013b-4843-a114-ca81996956ad\" (UID: \"0dd7617e-013b-4843-a114-ca81996956ad\") " Sep 10 20:11:39.648940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648566 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:39.648940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648696 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:39.648940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648734 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:39.648940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648867 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd7617e-013b-4843-a114-ca81996956ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:11:39.649212 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648940 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:39.649212 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648955 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.649212 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648976 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.649212 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.648991 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd7617e-013b-4843-a114-ca81996956ad-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.649212 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.649006 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.650556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.650535 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:11:39.650722 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.650703 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:39.650988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.650967 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-kube-api-access-zmdz8" (OuterVolumeSpecName: "kube-api-access-zmdz8") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "kube-api-access-zmdz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:11:39.651157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.650982 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0dd7617e-013b-4843-a114-ca81996956ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0dd7617e-013b-4843-a114-ca81996956ad" (UID: "0dd7617e-013b-4843-a114-ca81996956ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:11:39.750137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.750104 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.750137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.750130 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd7617e-013b-4843-a114-ca81996956ad-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.750137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.750140 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmdz8\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-kube-api-access-zmdz8\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.750415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.750148 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd7617e-013b-4843-a114-ca81996956ad-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:39.750415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:39.750159 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd7617e-013b-4843-a114-ca81996956ad-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:11:40.222137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.222095 2570 generic.go:356] "Generic (PLEG): container finished" podID="0dd7617e-013b-4843-a114-ca81996956ad" containerID="4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8" exitCode=0 Sep 10 20:11:40.222318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.222155 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" Sep 10 20:11:40.222318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.222178 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" event={"ID":"0dd7617e-013b-4843-a114-ca81996956ad","Type":"ContainerDied","Data":"4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8"} Sep 10 20:11:40.222318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.222263 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b" event={"ID":"0dd7617e-013b-4843-a114-ca81996956ad","Type":"ContainerDied","Data":"a1be6120c317291be9abd0035aba8fd8f797d32e911da2bdfdd4df5e4b211f8f"} Sep 10 20:11:40.222318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.222285 2570 scope.go:117] "RemoveContainer" containerID="4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8" Sep 10 20:11:40.233428 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.233409 2570 scope.go:117] "RemoveContainer" containerID="4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8" Sep 10 20:11:40.233722 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:11:40.233701 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8\": container with ID starting with 4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8 not found: ID does not exist" containerID="4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8" Sep 10 20:11:40.233795 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.233742 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8"} err="failed to get container status \"4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8\": rpc error: code = NotFound desc = could not find container \"4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8\": container with ID starting with 4a84e3cd102a73da51b510d1fc3de3118fa3d53d31c351feb0be890515dbafe8 not found: ID does not exist" Sep 10 20:11:40.249218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.249191 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b"] Sep 10 20:11:40.255587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:40.255563 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-842160-openshift-default-54df884cd-hqd2b"] Sep 10 20:11:41.766658 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:41.766626 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dd7617e-013b-4843-a114-ca81996956ad" path="/var/lib/kubelet/pods/0dd7617e-013b-4843-a114-ca81996956ad/volumes" Sep 10 20:11:45.545951 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.545863 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v"] Sep 10 20:11:45.546630 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.546608 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dd7617e-013b-4843-a114-ca81996956ad" containerName="istio-proxy" Sep 10 20:11:45.546695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.546636 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dd7617e-013b-4843-a114-ca81996956ad" containerName="istio-proxy" Sep 10 20:11:45.546759 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.546746 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dd7617e-013b-4843-a114-ca81996956ad" containerName="istio-proxy" Sep 10 20:11:45.551956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.551933 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.556377 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.556338 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-6550b4-openshift-default-dockercfg-ftf2h\"" Sep 10 20:11:45.595989 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.595952 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v"] Sep 10 20:11:45.715714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715681 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e618051e-29b7-4ede-be2a-659f5c484acd-istiod-ca-cert\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.715714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715719 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-certs\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.715946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz22c\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-kube-api-access-lz22c\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.715946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-envoy\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.715946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-data\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.715946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-istio-token\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.715946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715932 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-credential-socket\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.716164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.715979 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-socket\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.716164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.716037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e618051e-29b7-4ede-be2a-659f5c484acd-istio-podinfo\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.816749 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e618051e-29b7-4ede-be2a-659f5c484acd-istio-podinfo\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.816749 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816708 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e618051e-29b7-4ede-be2a-659f5c484acd-istiod-ca-cert\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.816749 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816738 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-certs\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816764 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz22c\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-kube-api-access-lz22c\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816789 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-envoy\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816814 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-data\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-istio-token\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.816942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-credential-socket\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.817013 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-socket\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.817260 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-certs\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817529 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.817506 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-data\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817606 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.817559 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-socket\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.817649 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.817628 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-credential-socket\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.818062 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.818040 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e618051e-29b7-4ede-be2a-659f5c484acd-istiod-ca-cert\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.819162 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.819142 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-envoy\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.819398 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.819378 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e618051e-29b7-4ede-be2a-659f5c484acd-istio-podinfo\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.831304 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.831273 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-istio-token\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.831437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.831407 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz22c\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-kube-api-access-lz22c\") pod \"e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:45.863341 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:45.863313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:46.146735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.146710 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v"] Sep 10 20:11:46.148791 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:11:46.148759 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode618051e_29b7_4ede_be2a_659f5c484acd.slice/crio-3d1e089bc39ad1f8624cda4343d7fc6e4c96e03c33d478723859339e2eb0ddce WatchSource:0}: Error finding container 3d1e089bc39ad1f8624cda4343d7fc6e4c96e03c33d478723859339e2eb0ddce: Status 404 returned error can't find the container with id 3d1e089bc39ad1f8624cda4343d7fc6e4c96e03c33d478723859339e2eb0ddce Sep 10 20:11:46.151059 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.151024 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:11:46.151173 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.151109 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:11:46.151173 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.151139 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:11:46.260456 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.260415 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" event={"ID":"e618051e-29b7-4ede-be2a-659f5c484acd","Type":"ContainerStarted","Data":"abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d"} Sep 10 20:11:46.260656 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.260473 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" event={"ID":"e618051e-29b7-4ede-be2a-659f5c484acd","Type":"ContainerStarted","Data":"3d1e089bc39ad1f8624cda4343d7fc6e4c96e03c33d478723859339e2eb0ddce"} Sep 10 20:11:46.285170 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.285119 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podStartSLOduration=1.285097041 podStartE2EDuration="1.285097041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:11:46.284881256 +0000 UTC m=+1075.005828377" watchObservedRunningTime="2026-09-10 20:11:46.285097041 +0000 UTC m=+1075.006044159" Sep 10 20:11:46.863453 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:46.863421 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:47.869859 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:47.869827 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:48.271985 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:48.271951 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:48.272978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:48.272956 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:11:50.673720 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.673680 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98"] Sep 10 20:11:50.677259 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.677238 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.681587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.681560 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-6550b4\"" Sep 10 20:11:50.683176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.683157 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-6550b4-dockercfg-qrsdp\"" Sep 10 20:11:50.700774 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.700740 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98"] Sep 10 20:11:50.773284 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.773242 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ef9b018-dae0-497b-9655-b940c5d142fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.773468 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.773327 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkpwn\" (UniqueName: \"kubernetes.io/projected/6ef9b018-dae0-497b-9655-b940c5d142fd-kube-api-access-rkpwn\") pod \"payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.873961 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.873927 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ef9b018-dae0-497b-9655-b940c5d142fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.874149 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.873984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkpwn\" (UniqueName: \"kubernetes.io/projected/6ef9b018-dae0-497b-9655-b940c5d142fd-kube-api-access-rkpwn\") pod \"payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.874637 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.874616 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ef9b018-dae0-497b-9655-b940c5d142fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.884765 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.884732 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkpwn\" (UniqueName: \"kubernetes.io/projected/6ef9b018-dae0-497b-9655-b940c5d142fd-kube-api-access-rkpwn\") pod \"payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:50.987894 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:50.987804 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:51.268406 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:51.268380 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98"] Sep 10 20:11:51.271518 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:11:51.271490 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef9b018_dae0_497b_9655_b940c5d142fd.slice/crio-2b83cfce81f3539011ea389a50116115e44def2f22876cfab3fa3631b01108cc WatchSource:0}: Error finding container 2b83cfce81f3539011ea389a50116115e44def2f22876cfab3fa3631b01108cc: Status 404 returned error can't find the container with id 2b83cfce81f3539011ea389a50116115e44def2f22876cfab3fa3631b01108cc Sep 10 20:11:51.289578 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:51.289544 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" event={"ID":"6ef9b018-dae0-497b-9655-b940c5d142fd","Type":"ContainerStarted","Data":"2b83cfce81f3539011ea389a50116115e44def2f22876cfab3fa3631b01108cc"} Sep 10 20:11:52.296609 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:52.296577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" event={"ID":"6ef9b018-dae0-497b-9655-b940c5d142fd","Type":"ContainerStarted","Data":"0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b"} Sep 10 20:11:52.297086 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:52.296693 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:11:52.319760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:11:52.319698 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" podStartSLOduration=2.319683709 podStartE2EDuration="2.319683709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:11:52.317331883 +0000 UTC m=+1081.038279002" watchObservedRunningTime="2026-09-10 20:11:52.319683709 +0000 UTC m=+1081.040630828" Sep 10 20:12:03.303757 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:03.303721 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:12:04.314250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:04.314205 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 20:12:04.314683 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:04.314284 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 20:12:05.374370 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:05.374333 2570 generic.go:356] "Generic (PLEG): container finished" podID="e618051e-29b7-4ede-be2a-659f5c484acd" containerID="abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d" exitCode=0 Sep 10 20:12:05.374928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:05.374397 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" event={"ID":"e618051e-29b7-4ede-be2a-659f5c484acd","Type":"ContainerDied","Data":"abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d"} Sep 10 20:12:05.374998 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:05.374961 2570 scope.go:117] "RemoveContainer" containerID="abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d" Sep 10 20:12:05.376090 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:05.376039 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:12:05.376157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:05.376126 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:12:05.376194 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:05.376159 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:12:06.381358 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:06.381326 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" event={"ID":"e618051e-29b7-4ede-be2a-659f5c484acd","Type":"ContainerStarted","Data":"3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740"} Sep 10 20:12:06.864439 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:06.864396 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:12:06.864707 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:06.864679 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 20:12:06.864763 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:06.864742 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 20:12:07.864227 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:07.864188 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 20:12:07.864675 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:07.864248 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 20:12:08.576497 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.576463 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k"] Sep 10 20:12:08.576697 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.576663 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" podUID="4ce090c0-6658-47e0-84d0-e3c171ef23da" containerName="payload-processing" containerID="cri-o://be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc" gracePeriod=30 Sep 10 20:12:08.635634 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.635602 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq"] Sep 10 20:12:08.635814 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.635795 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" podUID="95797dab-0df2-4dd4-b0f9-608e27a02ed3" containerName="payload-pre-processing" containerID="cri-o://5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65" gracePeriod=30 Sep 10 20:12:08.866369 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.865815 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 20:12:08.866369 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.865876 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 20:12:08.970133 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.970102 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:12:08.985754 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:08.985729 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.066481 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95797dab-0df2-4dd4-b0f9-608e27a02ed3-plugins-config-volume\") pod \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.066632 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8spt4\" (UniqueName: \"kubernetes.io/projected/95797dab-0df2-4dd4-b0f9-608e27a02ed3-kube-api-access-8spt4\") pod \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\" (UID: \"95797dab-0df2-4dd4-b0f9-608e27a02ed3\") " Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.066777 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f9p7\" (UniqueName: \"kubernetes.io/projected/4ce090c0-6658-47e0-84d0-e3c171ef23da-kube-api-access-8f9p7\") pod \"4ce090c0-6658-47e0-84d0-e3c171ef23da\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.068230 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ce090c0-6658-47e0-84d0-e3c171ef23da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ce090c0-6658-47e0-84d0-e3c171ef23da" (UID: "4ce090c0-6658-47e0-84d0-e3c171ef23da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.068298 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ce090c0-6658-47e0-84d0-e3c171ef23da-plugins-config-volume\") pod \"4ce090c0-6658-47e0-84d0-e3c171ef23da\" (UID: \"4ce090c0-6658-47e0-84d0-e3c171ef23da\") " Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.068677 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95797dab-0df2-4dd4-b0f9-608e27a02ed3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95797dab-0df2-4dd4-b0f9-608e27a02ed3" (UID: "95797dab-0df2-4dd4-b0f9-608e27a02ed3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.068940 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ce090c0-6658-47e0-84d0-e3c171ef23da-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:12:09.069179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.068960 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95797dab-0df2-4dd4-b0f9-608e27a02ed3-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:12:09.071588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.071557 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95797dab-0df2-4dd4-b0f9-608e27a02ed3-kube-api-access-8spt4" (OuterVolumeSpecName: "kube-api-access-8spt4") pod "95797dab-0df2-4dd4-b0f9-608e27a02ed3" (UID: "95797dab-0df2-4dd4-b0f9-608e27a02ed3"). InnerVolumeSpecName "kube-api-access-8spt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:12:09.072389 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.072360 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ce090c0-6658-47e0-84d0-e3c171ef23da-kube-api-access-8f9p7" (OuterVolumeSpecName: "kube-api-access-8f9p7") pod "4ce090c0-6658-47e0-84d0-e3c171ef23da" (UID: "4ce090c0-6658-47e0-84d0-e3c171ef23da"). InnerVolumeSpecName "kube-api-access-8f9p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:12:09.170010 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.169920 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8spt4\" (UniqueName: \"kubernetes.io/projected/95797dab-0df2-4dd4-b0f9-608e27a02ed3-kube-api-access-8spt4\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:12:09.170010 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.169950 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f9p7\" (UniqueName: \"kubernetes.io/projected/4ce090c0-6658-47e0-84d0-e3c171ef23da-kube-api-access-8f9p7\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:12:09.400122 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.400059 2570 generic.go:356] "Generic (PLEG): container finished" podID="4ce090c0-6658-47e0-84d0-e3c171ef23da" containerID="be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc" exitCode=0 Sep 10 20:12:09.400122 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.400102 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" event={"ID":"4ce090c0-6658-47e0-84d0-e3c171ef23da","Type":"ContainerDied","Data":"be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc"} Sep 10 20:12:09.400122 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.400122 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" Sep 10 20:12:09.400365 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.400164 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k" event={"ID":"4ce090c0-6658-47e0-84d0-e3c171ef23da","Type":"ContainerDied","Data":"c9c7275d9aa9ea0c1e9bf9239865248ed9fdfaf9543b793cffc0e280984529d1"} Sep 10 20:12:09.400365 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.400187 2570 scope.go:117] "RemoveContainer" containerID="be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc" Sep 10 20:12:09.401931 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.401910 2570 generic.go:356] "Generic (PLEG): container finished" podID="95797dab-0df2-4dd4-b0f9-608e27a02ed3" containerID="5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65" exitCode=0 Sep 10 20:12:09.402030 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.401944 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" Sep 10 20:12:09.402030 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.401981 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" event={"ID":"95797dab-0df2-4dd4-b0f9-608e27a02ed3","Type":"ContainerDied","Data":"5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65"} Sep 10 20:12:09.402030 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.402011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq" event={"ID":"95797dab-0df2-4dd4-b0f9-608e27a02ed3","Type":"ContainerDied","Data":"b8f9606649ef9722e078fdd7d4999819eeb6696496a4bee1631f733cba312b35"} Sep 10 20:12:09.410949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.410933 2570 scope.go:117] "RemoveContainer" containerID="be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc" Sep 10 20:12:09.411254 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:12:09.411232 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc\": container with ID starting with be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc not found: ID does not exist" containerID="be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc" Sep 10 20:12:09.411331 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.411262 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc"} err="failed to get container status \"be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc\": rpc error: code = NotFound desc = could not find container \"be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc\": container with ID starting with be661a3a38faefb15a6c6ec0ab84ee5274429e9f663173a6e22feee5c9fa9cbc not found: ID does not exist" Sep 10 20:12:09.411331 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.411280 2570 scope.go:117] "RemoveContainer" containerID="5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65" Sep 10 20:12:09.419579 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.419561 2570 scope.go:117] "RemoveContainer" containerID="5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65" Sep 10 20:12:09.419878 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:12:09.419858 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65\": container with ID starting with 5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65 not found: ID does not exist" containerID="5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65" Sep 10 20:12:09.419945 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.419884 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65"} err="failed to get container status \"5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65\": rpc error: code = NotFound desc = could not find container \"5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65\": container with ID starting with 5ed066a42a7d724b34ce9fc9f9b62389b17ba78b22f207abffe04842bf447a65 not found: ID does not exist" Sep 10 20:12:09.445051 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.444957 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k"] Sep 10 20:12:09.453589 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.453550 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2017ae11-f6bb7899c-9c44k"] Sep 10 20:12:09.479087 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.479021 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq"] Sep 10 20:12:09.492051 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.491996 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2017ae11-79dd59b9fd-2ktkq"] Sep 10 20:12:09.769484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.769454 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ce090c0-6658-47e0-84d0-e3c171ef23da" path="/var/lib/kubelet/pods/4ce090c0-6658-47e0-84d0-e3c171ef23da/volumes" Sep 10 20:12:09.769873 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.769860 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95797dab-0df2-4dd4-b0f9-608e27a02ed3" path="/var/lib/kubelet/pods/95797dab-0df2-4dd4-b0f9-608e27a02ed3/volumes" Sep 10 20:12:09.864876 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.864818 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 20:12:09.865101 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:09.864900 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 20:12:10.864371 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:10.864332 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 20:12:10.864773 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:10.864389 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 20:12:11.867375 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:11.867344 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:12:11.867800 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:11.867776 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:12:11.868506 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:11.868485 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:12:40.429110 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429057 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f"] Sep 10 20:12:40.429590 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429550 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ce090c0-6658-47e0-84d0-e3c171ef23da" containerName="payload-processing" Sep 10 20:12:40.429590 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429563 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ce090c0-6658-47e0-84d0-e3c171ef23da" containerName="payload-processing" Sep 10 20:12:40.429590 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429585 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95797dab-0df2-4dd4-b0f9-608e27a02ed3" containerName="payload-pre-processing" Sep 10 20:12:40.429590 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429591 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="95797dab-0df2-4dd4-b0f9-608e27a02ed3" containerName="payload-pre-processing" Sep 10 20:12:40.429736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429657 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ce090c0-6658-47e0-84d0-e3c171ef23da" containerName="payload-processing" Sep 10 20:12:40.429736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.429667 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="95797dab-0df2-4dd4-b0f9-608e27a02ed3" containerName="payload-pre-processing" Sep 10 20:12:40.431821 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.431804 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.434812 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.434793 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-214b09\"" Sep 10 20:12:40.442333 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.442309 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f"] Sep 10 20:12:40.481463 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.481432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx6lf\" (UniqueName: \"kubernetes.io/projected/9edc4159-b99e-4ca4-bb52-bf2401855dbf-kube-api-access-kx6lf\") pod \"payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.481615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.481482 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9edc4159-b99e-4ca4-bb52-bf2401855dbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.499646 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.499616 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj"] Sep 10 20:12:40.502253 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.502238 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.506258 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.506232 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-214b09-dockercfg-r8x86\"" Sep 10 20:12:40.517457 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.517435 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj"] Sep 10 20:12:40.583957 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.583923 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9edc4159-b99e-4ca4-bb52-bf2401855dbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.584164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.584016 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c62f98e4-92b8-4ad1-bc0a-feec991b2685-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.584164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.584048 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4bk8\" (UniqueName: \"kubernetes.io/projected/c62f98e4-92b8-4ad1-bc0a-feec991b2685-kube-api-access-s4bk8\") pod \"payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.584164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.584123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx6lf\" (UniqueName: \"kubernetes.io/projected/9edc4159-b99e-4ca4-bb52-bf2401855dbf-kube-api-access-kx6lf\") pod \"payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.584678 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.584661 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9edc4159-b99e-4ca4-bb52-bf2401855dbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.597291 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.597268 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx6lf\" (UniqueName: \"kubernetes.io/projected/9edc4159-b99e-4ca4-bb52-bf2401855dbf-kube-api-access-kx6lf\") pod \"payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.684750 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.684648 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c62f98e4-92b8-4ad1-bc0a-feec991b2685-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.684750 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.684705 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4bk8\" (UniqueName: \"kubernetes.io/projected/c62f98e4-92b8-4ad1-bc0a-feec991b2685-kube-api-access-s4bk8\") pod \"payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.685395 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.685373 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c62f98e4-92b8-4ad1-bc0a-feec991b2685-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.700233 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.700207 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4bk8\" (UniqueName: \"kubernetes.io/projected/c62f98e4-92b8-4ad1-bc0a-feec991b2685-kube-api-access-s4bk8\") pod \"payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:40.743028 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.742993 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:40.812316 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:40.812287 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:41.014433 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.014288 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f"] Sep 10 20:12:41.017460 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:12:41.017427 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9edc4159_b99e_4ca4_bb52_bf2401855dbf.slice/crio-ebb80053f4bd388d4c6c595d9314b197955ea9b7c4b7dab89a76f7ac0611fd2b WatchSource:0}: Error finding container ebb80053f4bd388d4c6c595d9314b197955ea9b7c4b7dab89a76f7ac0611fd2b: Status 404 returned error can't find the container with id ebb80053f4bd388d4c6c595d9314b197955ea9b7c4b7dab89a76f7ac0611fd2b Sep 10 20:12:41.098385 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.098358 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj"] Sep 10 20:12:41.100349 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:12:41.100322 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62f98e4_92b8_4ad1_bc0a_feec991b2685.slice/crio-a135bb4febee55dabe57599ce411b1f6383a4b8770697486095282970b088262 WatchSource:0}: Error finding container a135bb4febee55dabe57599ce411b1f6383a4b8770697486095282970b088262: Status 404 returned error can't find the container with id a135bb4febee55dabe57599ce411b1f6383a4b8770697486095282970b088262 Sep 10 20:12:41.587830 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.587793 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" event={"ID":"9edc4159-b99e-4ca4-bb52-bf2401855dbf","Type":"ContainerStarted","Data":"d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6"} Sep 10 20:12:41.588361 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.587937 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:12:41.588361 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.587979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" event={"ID":"9edc4159-b99e-4ca4-bb52-bf2401855dbf","Type":"ContainerStarted","Data":"ebb80053f4bd388d4c6c595d9314b197955ea9b7c4b7dab89a76f7ac0611fd2b"} Sep 10 20:12:41.589460 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.589436 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" event={"ID":"c62f98e4-92b8-4ad1-bc0a-feec991b2685","Type":"ContainerStarted","Data":"74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe"} Sep 10 20:12:41.589561 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.589484 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" event={"ID":"c62f98e4-92b8-4ad1-bc0a-feec991b2685","Type":"ContainerStarted","Data":"a135bb4febee55dabe57599ce411b1f6383a4b8770697486095282970b088262"} Sep 10 20:12:41.589561 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.589512 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:41.608359 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.608316 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" podStartSLOduration=1.6083029039999999 podStartE2EDuration="1.608302904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:12:41.605136293 +0000 UTC m=+1130.326083427" watchObservedRunningTime="2026-09-10 20:12:41.608302904 +0000 UTC m=+1130.329250023" Sep 10 20:12:41.625242 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:41.625197 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" podStartSLOduration=1.625180756 podStartE2EDuration="1.625180756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:12:41.622694999 +0000 UTC m=+1130.343642117" watchObservedRunningTime="2026-09-10 20:12:41.625180756 +0000 UTC m=+1130.346127878" Sep 10 20:12:52.596972 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:52.596941 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:12:52.597383 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:12:52.597009 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:13:04.060893 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.060855 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr"] Sep 10 20:13:04.064765 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.064744 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.070756 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.070734 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-6ef723-openshift-default-dockercfg-4fqxn\"" Sep 10 20:13:04.087539 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.087512 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr"] Sep 10 20:13:04.221054 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lfgc\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-kube-api-access-5lfgc\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221257 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221090 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-certs\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221257 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221165 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-credential-socket\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221257 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221205 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-data\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221382 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-token\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221382 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-socket\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221382 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221336 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-envoy\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221382 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-podinfo\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.221527 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.221411 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22e13592-856a-43cf-8076-4b8edb7cb84e-istiod-ca-cert\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322467 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-certs\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322467 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-credential-socket\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322467 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-data\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322745 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-token\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322745 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322520 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-socket\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322745 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322554 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-envoy\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322745 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-podinfo\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322931 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322766 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22e13592-856a-43cf-8076-4b8edb7cb84e-istiod-ca-cert\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322931 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lfgc\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-kube-api-access-5lfgc\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.322931 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.322839 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-credential-socket\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.323105 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.323042 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-certs\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.323157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.323094 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-socket\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.323192 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.323152 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-data\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.323567 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.323545 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22e13592-856a-43cf-8076-4b8edb7cb84e-istiod-ca-cert\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.325090 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.325049 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-envoy\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.325298 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.325280 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-podinfo\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.339872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.339847 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lfgc\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-kube-api-access-5lfgc\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.340496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.340478 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-token\") pod \"e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.375475 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.375446 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:04.649663 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.649637 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr"] Sep 10 20:13:04.652034 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:13:04.652005 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e13592_856a_43cf_8076_4b8edb7cb84e.slice/crio-bdab09058d032c639a1d56ac299999511a372a0f1c280c25ac32cc8437a971f4 WatchSource:0}: Error finding container bdab09058d032c639a1d56ac299999511a372a0f1c280c25ac32cc8437a971f4: Status 404 returned error can't find the container with id bdab09058d032c639a1d56ac299999511a372a0f1c280c25ac32cc8437a971f4 Sep 10 20:13:04.654273 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.654240 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:04.654339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.654309 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:04.654383 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.654341 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:04.712594 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:04.712566 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" event={"ID":"22e13592-856a-43cf-8076-4b8edb7cb84e","Type":"ContainerStarted","Data":"bdab09058d032c639a1d56ac299999511a372a0f1c280c25ac32cc8437a971f4"} Sep 10 20:13:05.719570 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:05.719534 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" event={"ID":"22e13592-856a-43cf-8076-4b8edb7cb84e","Type":"ContainerStarted","Data":"603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a"} Sep 10 20:13:05.748782 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:05.748731 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podStartSLOduration=1.7487107960000001 podStartE2EDuration="1.748710796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:13:05.747117256 +0000 UTC m=+1154.468064379" watchObservedRunningTime="2026-09-10 20:13:05.748710796 +0000 UTC m=+1154.469657914" Sep 10 20:13:06.375731 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:06.375695 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:06.381223 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:06.381201 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:06.725386 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:06.725290 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:06.726379 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:06.726360 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:25.835507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:25.835474 2570 generic.go:356] "Generic (PLEG): container finished" podID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerID="603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a" exitCode=0 Sep 10 20:13:25.835946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:25.835548 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" event={"ID":"22e13592-856a-43cf-8076-4b8edb7cb84e","Type":"ContainerDied","Data":"603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a"} Sep 10 20:13:25.835946 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:25.835924 2570 scope.go:117] "RemoveContainer" containerID="603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a" Sep 10 20:13:25.837104 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:25.837057 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:25.837226 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:25.837155 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:25.837226 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:25.837203 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:26.843521 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:26.843485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" event={"ID":"22e13592-856a-43cf-8076-4b8edb7cb84e","Type":"ContainerStarted","Data":"cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b"} Sep 10 20:13:27.376258 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:27.376225 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:27.376489 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:27.376469 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 20:13:27.376541 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:27.376518 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 20:13:28.376355 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:28.376318 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 20:13:28.376797 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:28.376389 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 20:13:29.376545 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:29.376509 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 20:13:29.376950 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:29.376593 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 20:13:30.376186 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:30.376151 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 20:13:30.376385 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:30.376216 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 20:13:31.376347 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:31.376308 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 20:13:31.376747 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:31.376364 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 20:13:32.377635 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:32.377591 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 20:13:32.378063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:32.377673 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 20:13:33.380343 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:33.380314 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:33.380764 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:33.380579 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:33.381490 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:33.381471 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:13:42.024025 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.023986 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l"] Sep 10 20:13:42.026966 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.026945 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.029685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.029660 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e88b3e4f-openshift-default-dockercfg-9zs97\"" Sep 10 20:13:42.042367 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.042338 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l"] Sep 10 20:13:42.203393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-data\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203410 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-socket\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203439 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-certs\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-credential-socket\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203549 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f92e9f-8821-483d-b36b-f5341eff9106-istiod-ca-cert\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203588 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-istio-token\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203645 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f92e9f-8821-483d-b36b-f5341eff9106-istio-podinfo\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203698 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clkn6\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-kube-api-access-clkn6\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.203760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.203729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-envoy\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305399 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-data\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305399 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305352 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-socket\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305399 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305388 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-certs\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305433 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-credential-socket\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f92e9f-8821-483d-b36b-f5341eff9106-istiod-ca-cert\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305485 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-istio-token\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305535 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f92e9f-8821-483d-b36b-f5341eff9106-istio-podinfo\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305585 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clkn6\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-kube-api-access-clkn6\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.305695 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-envoy\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.306002 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305736 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-data\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.306002 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305785 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-socket\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.306002 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.305873 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-credential-socket\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.306148 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.306120 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-certs\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.306218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.306199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f92e9f-8821-483d-b36b-f5341eff9106-istiod-ca-cert\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.307899 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.307876 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-envoy\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.308166 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.308148 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f92e9f-8821-483d-b36b-f5341eff9106-istio-podinfo\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.314461 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.314432 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clkn6\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-kube-api-access-clkn6\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.314656 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.314639 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-istio-token\") pod \"e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.337975 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.337951 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:42.617166 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.617134 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l"] Sep 10 20:13:42.618005 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:13:42.617980 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f92e9f_8821_483d_b36b_f5341eff9106.slice/crio-9f0eb3aa67cb5a1fae5987c3a12524b3d27b1bde3864fa56468dbc9eff3bb412 WatchSource:0}: Error finding container 9f0eb3aa67cb5a1fae5987c3a12524b3d27b1bde3864fa56468dbc9eff3bb412: Status 404 returned error can't find the container with id 9f0eb3aa67cb5a1fae5987c3a12524b3d27b1bde3864fa56468dbc9eff3bb412 Sep 10 20:13:42.620279 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.620244 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:42.620375 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.620309 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:42.620375 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.620340 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:13:42.942372 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.942274 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" event={"ID":"98f92e9f-8821-483d-b36b-f5341eff9106","Type":"ContainerStarted","Data":"2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150"} Sep 10 20:13:42.942372 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.942338 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" event={"ID":"98f92e9f-8821-483d-b36b-f5341eff9106","Type":"ContainerStarted","Data":"9f0eb3aa67cb5a1fae5987c3a12524b3d27b1bde3864fa56468dbc9eff3bb412"} Sep 10 20:13:42.965958 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:42.965907 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podStartSLOduration=1.965889948 podStartE2EDuration="1.965889948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:13:42.963341674 +0000 UTC m=+1191.684288796" watchObservedRunningTime="2026-09-10 20:13:42.965889948 +0000 UTC m=+1191.686837066" Sep 10 20:13:43.338849 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:43.338815 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:43.344156 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:43.344135 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:43.947203 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:43.947175 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:43.948019 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:43.947993 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:13:51.919512 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:13:51.919481 2570 scope.go:117] "RemoveContainer" containerID="eca11ec0eea20cdcfee5c4c2020033f0204ab7108d9abb1438c24ff37a2ea630" Sep 10 20:14:06.069902 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:06.069816 2570 generic.go:356] "Generic (PLEG): container finished" podID="98f92e9f-8821-483d-b36b-f5341eff9106" containerID="2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150" exitCode=0 Sep 10 20:14:06.070345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:06.069890 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" event={"ID":"98f92e9f-8821-483d-b36b-f5341eff9106","Type":"ContainerDied","Data":"2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150"} Sep 10 20:14:06.070399 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:06.070348 2570 scope.go:117] "RemoveContainer" containerID="2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150" Sep 10 20:14:06.071527 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:06.071499 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:14:06.071606 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:06.071573 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:14:06.071646 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:06.071620 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:14:07.077452 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:07.077419 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" event={"ID":"98f92e9f-8821-483d-b36b-f5341eff9106","Type":"ContainerStarted","Data":"553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50"} Sep 10 20:14:07.338649 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:07.338509 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:14:07.339243 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:07.339216 2570 patch_prober.go:28] interesting pod/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Sep 10 20:14:07.339455 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:07.339433 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Sep 10 20:14:08.339371 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:08.339329 2570 patch_prober.go:28] interesting pod/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Sep 10 20:14:08.339978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:08.339943 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Sep 10 20:14:09.339500 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:09.339034 2570 patch_prober.go:28] interesting pod/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Sep 10 20:14:09.339500 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:09.339125 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Sep 10 20:14:10.338591 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:10.338552 2570 patch_prober.go:28] interesting pod/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Sep 10 20:14:10.338889 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:10.338862 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Sep 10 20:14:11.351101 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:11.340550 2570 patch_prober.go:28] interesting pod/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Sep 10 20:14:11.351101 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:11.340631 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Sep 10 20:14:12.342393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:12.342361 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:14:12.342750 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:12.342726 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:14:12.343402 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:12.343379 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:14:59.376542 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.376501 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb"] Sep 10 20:14:59.379864 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.379843 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:14:59.382708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.382686 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:14:59.382839 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.382767 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:14:59.389940 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.389915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb"] Sep 10 20:14:59.420162 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.420123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/705662b0-b9d7-4d4e-86da-e6eb2179a7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb\" (UID: \"705662b0-b9d7-4d4e-86da-e6eb2179a7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:14:59.521147 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.521098 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/705662b0-b9d7-4d4e-86da-e6eb2179a7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb\" (UID: \"705662b0-b9d7-4d4e-86da-e6eb2179a7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:14:59.521753 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.521724 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/705662b0-b9d7-4d4e-86da-e6eb2179a7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb\" (UID: \"705662b0-b9d7-4d4e-86da-e6eb2179a7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:14:59.693008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.692909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:14:59.969284 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:14:59.969254 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb"] Sep 10 20:14:59.970808 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:14:59.970782 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705662b0_b9d7_4d4e_86da_e6eb2179a7dc.slice/crio-a255ac00b0199ff5c3efdc60eca5521db602911ac66499d9c26c39a737114718 WatchSource:0}: Error finding container a255ac00b0199ff5c3efdc60eca5521db602911ac66499d9c26c39a737114718: Status 404 returned error can't find the container with id a255ac00b0199ff5c3efdc60eca5521db602911ac66499d9c26c39a737114718 Sep 10 20:15:00.162707 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.162669 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk"] Sep 10 20:15:00.169281 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.169251 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2"] Sep 10 20:15:00.170125 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.170099 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" Sep 10 20:15:00.173889 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.173868 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" Sep 10 20:15:00.182392 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.182369 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" Sep 10 20:15:00.189441 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.189405 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2"] Sep 10 20:15:00.193952 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.192377 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" Sep 10 20:15:00.193952 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.193181 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk"] Sep 10 20:15:00.367259 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.367222 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" event={"ID":"705662b0-b9d7-4d4e-86da-e6eb2179a7dc","Type":"ContainerStarted","Data":"9b8243cfa368b48bde4f4b7f47ba4096ae3de1cd6d7a79f46c7663478e5832bf"} Sep 10 20:15:00.367460 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.367278 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" event={"ID":"705662b0-b9d7-4d4e-86da-e6eb2179a7dc","Type":"ContainerStarted","Data":"a255ac00b0199ff5c3efdc60eca5521db602911ac66499d9c26c39a737114718"} Sep 10 20:15:00.385352 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.385307 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" podStartSLOduration=1.3852915399999999 podStartE2EDuration="1.38529154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:15:00.383706069 +0000 UTC m=+1269.104653191" watchObservedRunningTime="2026-09-10 20:15:00.38529154 +0000 UTC m=+1269.106238671" Sep 10 20:15:00.474229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.474205 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk"] Sep 10 20:15:00.476116 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:15:00.476084 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28253cb_62f3_4abc_ab17_0cedc96e9c45.slice/crio-612f340aebe3edc6c1621e6664de63771f9facc53719ab76c5b4f34081176d39 WatchSource:0}: Error finding container 612f340aebe3edc6c1621e6664de63771f9facc53719ab76c5b4f34081176d39: Status 404 returned error can't find the container with id 612f340aebe3edc6c1621e6664de63771f9facc53719ab76c5b4f34081176d39 Sep 10 20:15:00.495139 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:00.495116 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2"] Sep 10 20:15:00.496992 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:15:00.496954 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125db6f3_d1bc_4f42_99b1_8f9cda9bf510.slice/crio-632e835df6718bde1113d7720626e21afa77bc4157f48821c93be1c0ba6067ca WatchSource:0}: Error finding container 632e835df6718bde1113d7720626e21afa77bc4157f48821c93be1c0ba6067ca: Status 404 returned error can't find the container with id 632e835df6718bde1113d7720626e21afa77bc4157f48821c93be1c0ba6067ca Sep 10 20:15:01.373884 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.373850 2570 generic.go:356] "Generic (PLEG): container finished" podID="705662b0-b9d7-4d4e-86da-e6eb2179a7dc" containerID="9b8243cfa368b48bde4f4b7f47ba4096ae3de1cd6d7a79f46c7663478e5832bf" exitCode=0 Sep 10 20:15:01.374089 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.373931 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" event={"ID":"705662b0-b9d7-4d4e-86da-e6eb2179a7dc","Type":"ContainerDied","Data":"9b8243cfa368b48bde4f4b7f47ba4096ae3de1cd6d7a79f46c7663478e5832bf"} Sep 10 20:15:01.375530 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.375505 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" event={"ID":"125db6f3-d1bc-4f42-99b1-8f9cda9bf510","Type":"ContainerStarted","Data":"d0bc00318d9b974c58087353220c4a650210496632ef4679bf644259bb816279"} Sep 10 20:15:01.375639 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.375551 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" event={"ID":"125db6f3-d1bc-4f42-99b1-8f9cda9bf510","Type":"ContainerStarted","Data":"632e835df6718bde1113d7720626e21afa77bc4157f48821c93be1c0ba6067ca"} Sep 10 20:15:01.376979 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.376959 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" event={"ID":"f28253cb-62f3-4abc-ab17-0cedc96e9c45","Type":"ContainerStarted","Data":"fa1357c787a002c34724f82e0d56abf4575bdc44b0e79289955939948c17c1cc"} Sep 10 20:15:01.377043 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.376995 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" event={"ID":"f28253cb-62f3-4abc-ab17-0cedc96e9c45","Type":"ContainerStarted","Data":"612f340aebe3edc6c1621e6664de63771f9facc53719ab76c5b4f34081176d39"} Sep 10 20:15:01.406982 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.406935 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" podStartSLOduration=1.406919706 podStartE2EDuration="1.406919706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:15:01.405963411 +0000 UTC m=+1270.126910531" watchObservedRunningTime="2026-09-10 20:15:01.406919706 +0000 UTC m=+1270.127866831" Sep 10 20:15:01.422354 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:01.422309 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" podStartSLOduration=1.4222934729999999 podStartE2EDuration="1.422293473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:15:01.419915285 +0000 UTC m=+1270.140862404" watchObservedRunningTime="2026-09-10 20:15:01.422293473 +0000 UTC m=+1270.143240591" Sep 10 20:15:02.384150 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.384113 2570 generic.go:356] "Generic (PLEG): container finished" podID="f28253cb-62f3-4abc-ab17-0cedc96e9c45" containerID="fa1357c787a002c34724f82e0d56abf4575bdc44b0e79289955939948c17c1cc" exitCode=0 Sep 10 20:15:02.384352 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.384206 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" event={"ID":"f28253cb-62f3-4abc-ab17-0cedc96e9c45","Type":"ContainerDied","Data":"fa1357c787a002c34724f82e0d56abf4575bdc44b0e79289955939948c17c1cc"} Sep 10 20:15:02.385701 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.385675 2570 generic.go:356] "Generic (PLEG): container finished" podID="125db6f3-d1bc-4f42-99b1-8f9cda9bf510" containerID="d0bc00318d9b974c58087353220c4a650210496632ef4679bf644259bb816279" exitCode=0 Sep 10 20:15:02.385830 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.385703 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" event={"ID":"125db6f3-d1bc-4f42-99b1-8f9cda9bf510","Type":"ContainerDied","Data":"d0bc00318d9b974c58087353220c4a650210496632ef4679bf644259bb816279"} Sep 10 20:15:02.539409 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.539381 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:15:02.552010 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.551984 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/705662b0-b9d7-4d4e-86da-e6eb2179a7dc-maas-api-service-ca\") pod \"705662b0-b9d7-4d4e-86da-e6eb2179a7dc\" (UID: \"705662b0-b9d7-4d4e-86da-e6eb2179a7dc\") " Sep 10 20:15:02.552329 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.552310 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705662b0-b9d7-4d4e-86da-e6eb2179a7dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "705662b0-b9d7-4d4e-86da-e6eb2179a7dc" (UID: "705662b0-b9d7-4d4e-86da-e6eb2179a7dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:15:02.653740 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:02.653651 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/705662b0-b9d7-4d4e-86da-e6eb2179a7dc-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:03.395284 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:03.395253 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" Sep 10 20:15:03.395495 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:03.395280 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb" event={"ID":"705662b0-b9d7-4d4e-86da-e6eb2179a7dc","Type":"ContainerDied","Data":"a255ac00b0199ff5c3efdc60eca5521db602911ac66499d9c26c39a737114718"} Sep 10 20:15:03.395495 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:03.395338 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a255ac00b0199ff5c3efdc60eca5521db602911ac66499d9c26c39a737114718" Sep 10 20:15:03.557886 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:03.557856 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" Sep 10 20:15:03.560780 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:03.560760 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" Sep 10 20:15:04.402809 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:04.402768 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" Sep 10 20:15:04.402809 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:04.402794 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817855-j7nd2" event={"ID":"125db6f3-d1bc-4f42-99b1-8f9cda9bf510","Type":"ContainerDied","Data":"632e835df6718bde1113d7720626e21afa77bc4157f48821c93be1c0ba6067ca"} Sep 10 20:15:04.403035 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:04.402873 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="632e835df6718bde1113d7720626e21afa77bc4157f48821c93be1c0ba6067ca" Sep 10 20:15:04.404257 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:04.404233 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" Sep 10 20:15:04.404406 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:04.404231 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk" event={"ID":"f28253cb-62f3-4abc-ab17-0cedc96e9c45","Type":"ContainerDied","Data":"612f340aebe3edc6c1621e6664de63771f9facc53719ab76c5b4f34081176d39"} Sep 10 20:15:04.404406 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:04.404364 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="612f340aebe3edc6c1621e6664de63771f9facc53719ab76c5b4f34081176d39" Sep 10 20:15:15.237996 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:15.237957 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb"] Sep 10 20:15:15.240239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:15.240214 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e88b3e4f-vhkxb"] Sep 10 20:15:15.768010 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:15.767970 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="705662b0-b9d7-4d4e-86da-e6eb2179a7dc" path="/var/lib/kubelet/pods/705662b0-b9d7-4d4e-86da-e6eb2179a7dc/volumes" Sep 10 20:15:50.079007 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:50.078972 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l"] Sep 10 20:15:50.079549 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:50.079200 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" containerID="cri-o://553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50" gracePeriod=30 Sep 10 20:15:51.950177 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:51.950147 2570 scope.go:117] "RemoveContainer" containerID="b9f2a2092206b775f7a4bb1914dfbd45ee244aa1beb813e662efa2dad597a69c" Sep 10 20:15:55.323139 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.323106 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:15:55.494735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.494711 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clkn6\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-kube-api-access-clkn6\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.494928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.494785 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f92e9f-8821-483d-b36b-f5341eff9106-istiod-ca-cert\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.494928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.494829 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f92e9f-8821-483d-b36b-f5341eff9106-istio-podinfo\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.494928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.494863 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-istio-token\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.494928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.494901 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-credential-socket\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.495196 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.494944 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-data\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.495196 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495000 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-envoy\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.495196 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495038 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-certs\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.495196 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495124 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-socket\") pod \"98f92e9f-8821-483d-b36b-f5341eff9106\" (UID: \"98f92e9f-8821-483d-b36b-f5341eff9106\") " Sep 10 20:15:55.495400 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495210 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:15:55.495400 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495260 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f92e9f-8821-483d-b36b-f5341eff9106-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:15:55.495509 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495454 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:15:55.495571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495488 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:15:55.495636 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495562 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-data" (OuterVolumeSpecName: "istio-data") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:15:55.495737 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495717 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f92e9f-8821-483d-b36b-f5341eff9106-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.495803 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495742 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.495803 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495758 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.495803 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495772 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.495803 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.495785 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.497148 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.497121 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98f92e9f-8821-483d-b36b-f5341eff9106-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:15:55.497267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.497188 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-kube-api-access-clkn6" (OuterVolumeSpecName: "kube-api-access-clkn6") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "kube-api-access-clkn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:15:55.497267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.497230 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-istio-token" (OuterVolumeSpecName: "istio-token") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:15:55.497349 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.497299 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98f92e9f-8821-483d-b36b-f5341eff9106" (UID: "98f92e9f-8821-483d-b36b-f5341eff9106"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:15:55.597201 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.597166 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f92e9f-8821-483d-b36b-f5341eff9106-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.597201 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.597195 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.597201 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.597204 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f92e9f-8821-483d-b36b-f5341eff9106-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.597435 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.597212 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clkn6\" (UniqueName: \"kubernetes.io/projected/98f92e9f-8821-483d-b36b-f5341eff9106-kube-api-access-clkn6\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:15:55.694160 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.694129 2570 generic.go:356] "Generic (PLEG): container finished" podID="98f92e9f-8821-483d-b36b-f5341eff9106" containerID="553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50" exitCode=0 Sep 10 20:15:55.694345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.694179 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" Sep 10 20:15:55.694345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.694180 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" event={"ID":"98f92e9f-8821-483d-b36b-f5341eff9106","Type":"ContainerDied","Data":"553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50"} Sep 10 20:15:55.694345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.694299 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l" event={"ID":"98f92e9f-8821-483d-b36b-f5341eff9106","Type":"ContainerDied","Data":"9f0eb3aa67cb5a1fae5987c3a12524b3d27b1bde3864fa56468dbc9eff3bb412"} Sep 10 20:15:55.694345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.694322 2570 scope.go:117] "RemoveContainer" containerID="553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50" Sep 10 20:15:55.703855 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.703840 2570 scope.go:117] "RemoveContainer" containerID="2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150" Sep 10 20:15:55.711938 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.711924 2570 scope.go:117] "RemoveContainer" containerID="553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50" Sep 10 20:15:55.712231 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:15:55.712212 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50\": container with ID starting with 553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50 not found: ID does not exist" containerID="553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50" Sep 10 20:15:55.712294 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.712239 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50"} err="failed to get container status \"553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50\": rpc error: code = NotFound desc = could not find container \"553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50\": container with ID starting with 553ecafd52ce6daa26a6f161c087887c66ce39b4d427857809dad00b5b2c4c50 not found: ID does not exist" Sep 10 20:15:55.712294 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.712257 2570 scope.go:117] "RemoveContainer" containerID="2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150" Sep 10 20:15:55.712475 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:15:55.712456 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150\": container with ID starting with 2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150 not found: ID does not exist" containerID="2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150" Sep 10 20:15:55.712530 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.712484 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150"} err="failed to get container status \"2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150\": rpc error: code = NotFound desc = could not find container \"2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150\": container with ID starting with 2890a213980d71ae120e2223643ae2088de964392ecc7278f17fefb0707f2150 not found: ID does not exist" Sep 10 20:15:55.715818 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.715794 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l"] Sep 10 20:15:55.719799 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.719775 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e88b3e4f-openshift-default-658cf99796-99b6l"] Sep 10 20:15:55.766215 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:15:55.766145 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" path="/var/lib/kubelet/pods/98f92e9f-8821-483d-b36b-f5341eff9106/volumes" Sep 10 20:16:44.683384 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.683341 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb"] Sep 10 20:16:44.684099 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684051 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" Sep 10 20:16:44.684181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684102 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" Sep 10 20:16:44.684181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684127 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" Sep 10 20:16:44.684181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684138 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" Sep 10 20:16:44.684181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684172 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="705662b0-b9d7-4d4e-86da-e6eb2179a7dc" containerName="revoke-keys" Sep 10 20:16:44.684181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684182 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="705662b0-b9d7-4d4e-86da-e6eb2179a7dc" containerName="revoke-keys" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684196 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28253cb-62f3-4abc-ab17-0cedc96e9c45" containerName="cleanup" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684204 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28253cb-62f3-4abc-ab17-0cedc96e9c45" containerName="cleanup" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684229 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="125db6f3-d1bc-4f42-99b1-8f9cda9bf510" containerName="cleanup" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684237 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="125db6f3-d1bc-4f42-99b1-8f9cda9bf510" containerName="cleanup" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684349 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f28253cb-62f3-4abc-ab17-0cedc96e9c45" containerName="cleanup" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684362 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684372 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="705662b0-b9d7-4d4e-86da-e6eb2179a7dc" containerName="revoke-keys" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684382 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="125db6f3-d1bc-4f42-99b1-8f9cda9bf510" containerName="cleanup" Sep 10 20:16:44.684436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.684391 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f92e9f-8821-483d-b36b-f5341eff9106" containerName="istio-proxy" Sep 10 20:16:44.687189 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.687160 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.690326 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.690307 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-bd3eab-openshift-default-dockercfg-pmdfz\"" Sep 10 20:16:44.698852 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.698829 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb"] Sep 10 20:16:44.875267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875223 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-data\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875454 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875320 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7rx8\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-kube-api-access-l7rx8\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875454 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83269840-768a-425a-9244-89638d4c566e-istio-podinfo\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875454 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875428 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-socket\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875568 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875466 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-credential-socket\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875568 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-istio-token\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875633 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875594 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-certs\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875633 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875619 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-envoy\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.875700 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.875665 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83269840-768a-425a-9244-89638d4c566e-istiod-ca-cert\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976373 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-credential-socket\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-istio-token\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976477 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-certs\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-envoy\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83269840-768a-425a-9244-89638d4c566e-istiod-ca-cert\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-data\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976627 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7rx8\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-kube-api-access-l7rx8\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.976708 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83269840-768a-425a-9244-89638d4c566e-istio-podinfo\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.977048 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976738 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-socket\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.977048 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976844 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-certs\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.977048 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.976971 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-data\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.977250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.977117 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-socket\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.977250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.977145 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-credential-socket\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.977527 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.977508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83269840-768a-425a-9244-89638d4c566e-istiod-ca-cert\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.978947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.978922 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-envoy\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.979112 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.979091 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83269840-768a-425a-9244-89638d4c566e-istio-podinfo\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.984436 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.984411 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7rx8\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-kube-api-access-l7rx8\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.984906 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.984878 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-istio-token\") pod \"e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:44.998507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:44.998481 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:45.280840 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.280803 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb"] Sep 10 20:16:45.286209 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:16:45.286176 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83269840_768a_425a_9244_89638d4c566e.slice/crio-62894f375d3b797886ea56e7dc76fe82c69f5560a1133a21626f3e9a3d9910bf WatchSource:0}: Error finding container 62894f375d3b797886ea56e7dc76fe82c69f5560a1133a21626f3e9a3d9910bf: Status 404 returned error can't find the container with id 62894f375d3b797886ea56e7dc76fe82c69f5560a1133a21626f3e9a3d9910bf Sep 10 20:16:45.289341 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.289296 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:16:45.289458 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.289383 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:16:45.289458 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.289427 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:16:45.967023 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.966988 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" event={"ID":"83269840-768a-425a-9244-89638d4c566e","Type":"ContainerStarted","Data":"9a0add81ffbd9e5edca708ad29604fa3e21f3b76b59053af48d987a7d7e4b369"} Sep 10 20:16:45.967449 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.967045 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" event={"ID":"83269840-768a-425a-9244-89638d4c566e","Type":"ContainerStarted","Data":"62894f375d3b797886ea56e7dc76fe82c69f5560a1133a21626f3e9a3d9910bf"} Sep 10 20:16:45.989415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.989365 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podStartSLOduration=1.989347883 podStartE2EDuration="1.989347883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:16:45.987281865 +0000 UTC m=+1374.708228984" watchObservedRunningTime="2026-09-10 20:16:45.989347883 +0000 UTC m=+1374.710295002" Sep 10 20:16:45.999207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:45.999183 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:47.004413 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:47.004382 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:47.978218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:47.978189 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:47.979114 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:47.979091 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:16:49.867601 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.867563 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt"] Sep 10 20:16:49.877421 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.877393 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:49.880361 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.880333 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-bd3eab\"" Sep 10 20:16:49.880783 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.880758 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-bd3eab-dockercfg-gb4fd\"" Sep 10 20:16:49.883239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.883214 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt"] Sep 10 20:16:49.933014 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.932976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gk7h\" (UniqueName: \"kubernetes.io/projected/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-kube-api-access-6gk7h\") pod \"payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:49.933229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:49.933201 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:50.034372 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.034339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gk7h\" (UniqueName: \"kubernetes.io/projected/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-kube-api-access-6gk7h\") pod \"payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:50.034527 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.034455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:50.035218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.035196 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:50.046582 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.046550 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gk7h\" (UniqueName: \"kubernetes.io/projected/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-kube-api-access-6gk7h\") pod \"payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:50.191642 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.191555 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:50.469551 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.469521 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt"] Sep 10 20:16:50.471229 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:16:50.471198 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f9f70a_8b26_4328_8ca3_47bdaaf1641c.slice/crio-4688d696b1d9f7760e856d10ab4f4d4aa8fa6e52a86f2401a004ddb72bf07c6b WatchSource:0}: Error finding container 4688d696b1d9f7760e856d10ab4f4d4aa8fa6e52a86f2401a004ddb72bf07c6b: Status 404 returned error can't find the container with id 4688d696b1d9f7760e856d10ab4f4d4aa8fa6e52a86f2401a004ddb72bf07c6b Sep 10 20:16:50.996939 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.996902 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" event={"ID":"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c","Type":"ContainerStarted","Data":"513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e"} Sep 10 20:16:50.997362 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.996955 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" event={"ID":"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c","Type":"ContainerStarted","Data":"4688d696b1d9f7760e856d10ab4f4d4aa8fa6e52a86f2401a004ddb72bf07c6b"} Sep 10 20:16:50.997362 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:50.997108 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:16:51.017581 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:16:51.017536 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" podStartSLOduration=2.017523653 podStartE2EDuration="2.017523653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:16:51.01499504 +0000 UTC m=+1379.735942162" watchObservedRunningTime="2026-09-10 20:16:51.017523653 +0000 UTC m=+1379.738470773" Sep 10 20:17:02.006189 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:02.006157 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:17:04.019329 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:04.019280 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 20:17:04.019738 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:04.019377 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 20:17:07.103137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:07.103102 2570 generic.go:356] "Generic (PLEG): container finished" podID="83269840-768a-425a-9244-89638d4c566e" containerID="9a0add81ffbd9e5edca708ad29604fa3e21f3b76b59053af48d987a7d7e4b369" exitCode=0 Sep 10 20:17:07.103571 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:07.103169 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" event={"ID":"83269840-768a-425a-9244-89638d4c566e","Type":"ContainerDied","Data":"9a0add81ffbd9e5edca708ad29604fa3e21f3b76b59053af48d987a7d7e4b369"} Sep 10 20:17:07.103613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:07.103601 2570 scope.go:117] "RemoveContainer" containerID="9a0add81ffbd9e5edca708ad29604fa3e21f3b76b59053af48d987a7d7e4b369" Sep 10 20:17:07.104864 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:07.104837 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:17:07.104943 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:07.104918 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:17:07.104984 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:07.104949 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:17:08.110954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:08.110920 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" event={"ID":"83269840-768a-425a-9244-89638d4c566e","Type":"ContainerStarted","Data":"8e0a3edfcdebc5abc423a7b3bbe57be3ab3763c12fcecf45e82e668ced6d1483"} Sep 10 20:17:08.999274 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:08.999232 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:17:08.999493 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:08.999468 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 20:17:08.999543 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:08.999520 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 20:17:09.999008 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:09.998965 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 20:17:09.999510 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:09.999029 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 20:17:10.998860 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:10.998827 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 20:17:10.999049 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:10.998881 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 20:17:11.999852 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:11.999810 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 20:17:12.000389 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:11.999880 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 20:17:12.999396 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:12.999358 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 20:17:12.999579 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:12.999421 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 20:17:13.999700 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:13.999667 2570 patch_prober.go:28] interesting pod/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 20:17:14.000140 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:13.999733 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 20:17:15.003758 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:15.003726 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:17:15.004231 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:15.004035 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:17:15.004897 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:15.004871 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:17:44.850768 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:44.850682 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8"] Sep 10 20:17:44.853410 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:44.853391 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:44.856138 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:44.856116 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:17:44.856229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:44.856164 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:17:44.862460 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:44.862439 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8"] Sep 10 20:17:45.006272 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:45.006237 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cefa9ae1-2cb7-425e-9f12-1e6ce3126414-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8\" (UID: \"cefa9ae1-2cb7-425e-9f12-1e6ce3126414\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:45.107389 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:45.107298 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cefa9ae1-2cb7-425e-9f12-1e6ce3126414-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8\" (UID: \"cefa9ae1-2cb7-425e-9f12-1e6ce3126414\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:45.107945 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:45.107925 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cefa9ae1-2cb7-425e-9f12-1e6ce3126414-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8\" (UID: \"cefa9ae1-2cb7-425e-9f12-1e6ce3126414\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:45.165973 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:45.165938 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:45.435653 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:45.435625 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8"] Sep 10 20:17:45.438045 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:17:45.438013 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcefa9ae1_2cb7_425e_9f12_1e6ce3126414.slice/crio-f395cd4753bb7947d46d394b88e8d47584d708bf92904c963a8fc968cb92ca6b WatchSource:0}: Error finding container f395cd4753bb7947d46d394b88e8d47584d708bf92904c963a8fc968cb92ca6b: Status 404 returned error can't find the container with id f395cd4753bb7947d46d394b88e8d47584d708bf92904c963a8fc968cb92ca6b Sep 10 20:17:46.339632 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:46.339599 2570 generic.go:356] "Generic (PLEG): container finished" podID="cefa9ae1-2cb7-425e-9f12-1e6ce3126414" containerID="0fd61219827b30009870c653579f40b6a9ba2a1e1ccff06898bbc93ba8a253a7" exitCode=0 Sep 10 20:17:46.340032 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:46.339646 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" event={"ID":"cefa9ae1-2cb7-425e-9f12-1e6ce3126414","Type":"ContainerDied","Data":"0fd61219827b30009870c653579f40b6a9ba2a1e1ccff06898bbc93ba8a253a7"} Sep 10 20:17:46.340032 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:46.339712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" event={"ID":"cefa9ae1-2cb7-425e-9f12-1e6ce3126414","Type":"ContainerStarted","Data":"f395cd4753bb7947d46d394b88e8d47584d708bf92904c963a8fc968cb92ca6b"} Sep 10 20:17:47.505856 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:47.505825 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:47.633738 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:47.633646 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cefa9ae1-2cb7-425e-9f12-1e6ce3126414-maas-api-service-ca\") pod \"cefa9ae1-2cb7-425e-9f12-1e6ce3126414\" (UID: \"cefa9ae1-2cb7-425e-9f12-1e6ce3126414\") " Sep 10 20:17:47.634012 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:47.633989 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cefa9ae1-2cb7-425e-9f12-1e6ce3126414-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cefa9ae1-2cb7-425e-9f12-1e6ce3126414" (UID: "cefa9ae1-2cb7-425e-9f12-1e6ce3126414"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:17:47.735281 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:47.735241 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cefa9ae1-2cb7-425e-9f12-1e6ce3126414-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:17:48.352697 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:48.352662 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" Sep 10 20:17:48.352697 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:48.352680 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8" event={"ID":"cefa9ae1-2cb7-425e-9f12-1e6ce3126414","Type":"ContainerDied","Data":"f395cd4753bb7947d46d394b88e8d47584d708bf92904c963a8fc968cb92ca6b"} Sep 10 20:17:48.352925 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:17:48.352729 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f395cd4753bb7947d46d394b88e8d47584d708bf92904c963a8fc968cb92ca6b" Sep 10 20:18:00.397152 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.397115 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt"] Sep 10 20:18:00.397555 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.397374 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" podUID="f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" containerName="payload-processing" containerID="cri-o://513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e" gracePeriod=30 Sep 10 20:18:00.559827 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.559775 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8"] Sep 10 20:18:00.562761 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.562730 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd3eab-scgn8"] Sep 10 20:18:00.665131 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.665100 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:18:00.769282 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.769253 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-plugins-config-volume\") pod \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " Sep 10 20:18:00.769465 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.769331 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gk7h\" (UniqueName: \"kubernetes.io/projected/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-kube-api-access-6gk7h\") pod \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\" (UID: \"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c\") " Sep 10 20:18:00.769741 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.769711 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" (UID: "f0f9f70a-8b26-4328-8ca3-47bdaaf1641c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:00.771552 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.771529 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-kube-api-access-6gk7h" (OuterVolumeSpecName: "kube-api-access-6gk7h") pod "f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" (UID: "f0f9f70a-8b26-4328-8ca3-47bdaaf1641c"). InnerVolumeSpecName "kube-api-access-6gk7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:18:00.870609 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.870561 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:00.870609 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:00.870601 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gk7h\" (UniqueName: \"kubernetes.io/projected/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c-kube-api-access-6gk7h\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:01.434387 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.434346 2570 generic.go:356] "Generic (PLEG): container finished" podID="f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" containerID="513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e" exitCode=0 Sep 10 20:18:01.434387 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.434384 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" Sep 10 20:18:01.434917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.434419 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" event={"ID":"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c","Type":"ContainerDied","Data":"513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e"} Sep 10 20:18:01.434917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.434476 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt" event={"ID":"f0f9f70a-8b26-4328-8ca3-47bdaaf1641c","Type":"ContainerDied","Data":"4688d696b1d9f7760e856d10ab4f4d4aa8fa6e52a86f2401a004ddb72bf07c6b"} Sep 10 20:18:01.434917 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.434501 2570 scope.go:117] "RemoveContainer" containerID="513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e" Sep 10 20:18:01.446526 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.446505 2570 scope.go:117] "RemoveContainer" containerID="513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e" Sep 10 20:18:01.446872 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:18:01.446845 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e\": container with ID starting with 513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e not found: ID does not exist" containerID="513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e" Sep 10 20:18:01.446974 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.446883 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e"} err="failed to get container status \"513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e\": rpc error: code = NotFound desc = could not find container \"513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e\": container with ID starting with 513fba73ecab68249f8d8f54f3d87e41b8758d467666c2f1750771572007773e not found: ID does not exist" Sep 10 20:18:01.461496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.461467 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt"] Sep 10 20:18:01.463626 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.463604 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bd3eab-d554cb88f-qrzdt"] Sep 10 20:18:01.766899 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.766856 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cefa9ae1-2cb7-425e-9f12-1e6ce3126414" path="/var/lib/kubelet/pods/cefa9ae1-2cb7-425e-9f12-1e6ce3126414/volumes" Sep 10 20:18:01.767390 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:01.767370 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" path="/var/lib/kubelet/pods/f0f9f70a-8b26-4328-8ca3-47bdaaf1641c/volumes" Sep 10 20:18:07.325685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.325650 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb"] Sep 10 20:18:07.326247 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.325902 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" containerID="cri-o://8e0a3edfcdebc5abc423a7b3bbe57be3ab3763c12fcecf45e82e668ced6d1483" gracePeriod=30 Sep 10 20:18:07.693120 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693022 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6"] Sep 10 20:18:07.693495 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693482 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cefa9ae1-2cb7-425e-9f12-1e6ce3126414" containerName="revoke-keys" Sep 10 20:18:07.693540 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693497 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cefa9ae1-2cb7-425e-9f12-1e6ce3126414" containerName="revoke-keys" Sep 10 20:18:07.693540 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693522 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" containerName="payload-processing" Sep 10 20:18:07.693540 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693528 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" containerName="payload-processing" Sep 10 20:18:07.693649 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693592 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cefa9ae1-2cb7-425e-9f12-1e6ce3126414" containerName="revoke-keys" Sep 10 20:18:07.693649 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.693601 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0f9f70a-8b26-4328-8ca3-47bdaaf1641c" containerName="payload-processing" Sep 10 20:18:07.696578 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.696561 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:07.699207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.699179 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:18:07.699335 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.699181 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:18:07.705222 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.705197 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6"] Sep 10 20:18:07.839503 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.839467 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/260272c5-ca2e-4cbd-bd2c-de6eed6787ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6\" (UID: \"260272c5-ca2e-4cbd-bd2c-de6eed6787ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:07.940539 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.940504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/260272c5-ca2e-4cbd-bd2c-de6eed6787ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6\" (UID: \"260272c5-ca2e-4cbd-bd2c-de6eed6787ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:07.941379 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:07.941349 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/260272c5-ca2e-4cbd-bd2c-de6eed6787ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6\" (UID: \"260272c5-ca2e-4cbd-bd2c-de6eed6787ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:08.008831 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:08.008798 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:08.279586 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:08.279557 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6"] Sep 10 20:18:08.281130 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:18:08.281099 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260272c5_ca2e_4cbd_bd2c_de6eed6787ee.slice/crio-3358468552e491da49d1c0c674bc494927134fcaea40d4025e91a7560086d6dc WatchSource:0}: Error finding container 3358468552e491da49d1c0c674bc494927134fcaea40d4025e91a7560086d6dc: Status 404 returned error can't find the container with id 3358468552e491da49d1c0c674bc494927134fcaea40d4025e91a7560086d6dc Sep 10 20:18:08.481295 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:08.481258 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" event={"ID":"260272c5-ca2e-4cbd-bd2c-de6eed6787ee","Type":"ContainerStarted","Data":"4e03915b46fb2c4137e764bdf0fa59050c25f18bb1b5331932ef75c5d3fc1524"} Sep 10 20:18:08.481700 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:08.481316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" event={"ID":"260272c5-ca2e-4cbd-bd2c-de6eed6787ee","Type":"ContainerStarted","Data":"3358468552e491da49d1c0c674bc494927134fcaea40d4025e91a7560086d6dc"} Sep 10 20:18:08.498959 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:08.498911 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" podStartSLOduration=1.498894762 podStartE2EDuration="1.498894762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:18:08.496253765 +0000 UTC m=+1457.217200897" watchObservedRunningTime="2026-09-10 20:18:08.498894762 +0000 UTC m=+1457.219841881" Sep 10 20:18:09.488784 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:09.488752 2570 generic.go:356] "Generic (PLEG): container finished" podID="260272c5-ca2e-4cbd-bd2c-de6eed6787ee" containerID="4e03915b46fb2c4137e764bdf0fa59050c25f18bb1b5331932ef75c5d3fc1524" exitCode=0 Sep 10 20:18:09.488784 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:09.488794 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" event={"ID":"260272c5-ca2e-4cbd-bd2c-de6eed6787ee","Type":"ContainerDied","Data":"4e03915b46fb2c4137e764bdf0fa59050c25f18bb1b5331932ef75c5d3fc1524"} Sep 10 20:18:10.638297 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:10.638274 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:10.768341 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:10.768254 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/260272c5-ca2e-4cbd-bd2c-de6eed6787ee-maas-api-service-ca\") pod \"260272c5-ca2e-4cbd-bd2c-de6eed6787ee\" (UID: \"260272c5-ca2e-4cbd-bd2c-de6eed6787ee\") " Sep 10 20:18:10.768590 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:10.768566 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/260272c5-ca2e-4cbd-bd2c-de6eed6787ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "260272c5-ca2e-4cbd-bd2c-de6eed6787ee" (UID: "260272c5-ca2e-4cbd-bd2c-de6eed6787ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:10.869730 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:10.869682 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/260272c5-ca2e-4cbd-bd2c-de6eed6787ee-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:11.501098 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:11.501043 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" Sep 10 20:18:11.501296 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:11.501104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6" event={"ID":"260272c5-ca2e-4cbd-bd2c-de6eed6787ee","Type":"ContainerDied","Data":"3358468552e491da49d1c0c674bc494927134fcaea40d4025e91a7560086d6dc"} Sep 10 20:18:11.501296 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:11.501211 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3358468552e491da49d1c0c674bc494927134fcaea40d4025e91a7560086d6dc" Sep 10 20:18:12.508981 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.508947 2570 generic.go:356] "Generic (PLEG): container finished" podID="83269840-768a-425a-9244-89638d4c566e" containerID="8e0a3edfcdebc5abc423a7b3bbe57be3ab3763c12fcecf45e82e668ced6d1483" exitCode=0 Sep 10 20:18:12.509430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.509024 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" event={"ID":"83269840-768a-425a-9244-89638d4c566e","Type":"ContainerDied","Data":"8e0a3edfcdebc5abc423a7b3bbe57be3ab3763c12fcecf45e82e668ced6d1483"} Sep 10 20:18:12.509430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.509154 2570 scope.go:117] "RemoveContainer" containerID="9a0add81ffbd9e5edca708ad29604fa3e21f3b76b59053af48d987a7d7e4b369" Sep 10 20:18:12.587633 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.587571 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:18:12.685759 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685723 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-envoy\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.685956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685795 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-data\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.685956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685836 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83269840-768a-425a-9244-89638d4c566e-istio-podinfo\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.685956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685870 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7rx8\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-kube-api-access-l7rx8\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.685956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685925 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-certs\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.685956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685950 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83269840-768a-425a-9244-89638d4c566e-istiod-ca-cert\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.686278 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.685979 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-istio-token\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.686278 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686065 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-socket\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.686278 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686102 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-data" (OuterVolumeSpecName: "istio-data") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:18:12.686278 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686134 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-credential-socket\") pod \"83269840-768a-425a-9244-89638d4c566e\" (UID: \"83269840-768a-425a-9244-89638d4c566e\") " Sep 10 20:18:12.686479 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686327 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:18:12.686479 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686462 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:18:12.686778 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686754 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.686875 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686782 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.686875 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686797 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.686875 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.686813 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83269840-768a-425a-9244-89638d4c566e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:12.687032 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.687011 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:18:12.688204 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.688182 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:18:12.688339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.688318 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83269840-768a-425a-9244-89638d4c566e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:18:12.688339 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.688329 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-kube-api-access-l7rx8" (OuterVolumeSpecName: "kube-api-access-l7rx8") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "kube-api-access-l7rx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:18:12.688422 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.688332 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-istio-token" (OuterVolumeSpecName: "istio-token") pod "83269840-768a-425a-9244-89638d4c566e" (UID: "83269840-768a-425a-9244-89638d4c566e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:18:12.787526 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.787489 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83269840-768a-425a-9244-89638d4c566e-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.787526 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.787518 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.787526 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.787530 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.787765 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.787540 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83269840-768a-425a-9244-89638d4c566e-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.787765 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.787553 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83269840-768a-425a-9244-89638d4c566e-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:12.787765 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:12.787563 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7rx8\" (UniqueName: \"kubernetes.io/projected/83269840-768a-425a-9244-89638d4c566e-kube-api-access-l7rx8\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:13.516362 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:13.516335 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" Sep 10 20:18:13.516362 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:13.516353 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb" event={"ID":"83269840-768a-425a-9244-89638d4c566e","Type":"ContainerDied","Data":"62894f375d3b797886ea56e7dc76fe82c69f5560a1133a21626f3e9a3d9910bf"} Sep 10 20:18:13.516831 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:13.516408 2570 scope.go:117] "RemoveContainer" containerID="8e0a3edfcdebc5abc423a7b3bbe57be3ab3763c12fcecf45e82e668ced6d1483" Sep 10 20:18:13.538209 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:13.538180 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb"] Sep 10 20:18:13.542328 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:13.542296 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-bd3eab-openshift-default-d78f466d6-xv4xb"] Sep 10 20:18:13.766910 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:13.766817 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83269840-768a-425a-9244-89638d4c566e" path="/var/lib/kubelet/pods/83269840-768a-425a-9244-89638d4c566e/volumes" Sep 10 20:18:23.359429 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:23.359395 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6"] Sep 10 20:18:23.362117 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:23.362084 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-84c19e-4fjj6"] Sep 10 20:18:23.766135 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:23.766063 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="260272c5-ca2e-4cbd-bd2c-de6eed6787ee" path="/var/lib/kubelet/pods/260272c5-ca2e-4cbd-bd2c-de6eed6787ee/volumes" Sep 10 20:18:30.184870 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.184833 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t"] Sep 10 20:18:30.185345 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185330 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" Sep 10 20:18:30.185393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185347 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" Sep 10 20:18:30.185393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185361 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" Sep 10 20:18:30.185393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185366 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" Sep 10 20:18:30.185393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185388 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="260272c5-ca2e-4cbd-bd2c-de6eed6787ee" containerName="revoke-keys" Sep 10 20:18:30.185393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185395 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="260272c5-ca2e-4cbd-bd2c-de6eed6787ee" containerName="revoke-keys" Sep 10 20:18:30.185556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185471 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" Sep 10 20:18:30.185556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185480 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="83269840-768a-425a-9244-89638d4c566e" containerName="istio-proxy" Sep 10 20:18:30.185556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.185486 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="260272c5-ca2e-4cbd-bd2c-de6eed6787ee" containerName="revoke-keys" Sep 10 20:18:30.188165 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.188149 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:30.190759 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.190733 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:18:30.190905 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.190773 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:18:30.196699 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.196675 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t"] Sep 10 20:18:30.251575 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.251525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53bd00c-7317-49be-a7e1-f31cb6401abb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t\" (UID: \"f53bd00c-7317-49be-a7e1-f31cb6401abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:30.352512 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.352466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53bd00c-7317-49be-a7e1-f31cb6401abb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t\" (UID: \"f53bd00c-7317-49be-a7e1-f31cb6401abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:30.353165 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.353139 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53bd00c-7317-49be-a7e1-f31cb6401abb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t\" (UID: \"f53bd00c-7317-49be-a7e1-f31cb6401abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:30.499951 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.499908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:30.773469 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:30.773440 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t"] Sep 10 20:18:30.776456 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:18:30.776432 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53bd00c_7317_49be_a7e1_f31cb6401abb.slice/crio-d7f8552df5698ae381d5b14f186c1aebe717eadc98dbad9e8010ffb8a2c5c6ee WatchSource:0}: Error finding container d7f8552df5698ae381d5b14f186c1aebe717eadc98dbad9e8010ffb8a2c5c6ee: Status 404 returned error can't find the container with id d7f8552df5698ae381d5b14f186c1aebe717eadc98dbad9e8010ffb8a2c5c6ee Sep 10 20:18:31.626837 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:31.626806 2570 generic.go:356] "Generic (PLEG): container finished" podID="f53bd00c-7317-49be-a7e1-f31cb6401abb" containerID="31d34ee49b9756786d4b60800041852c23bbe0395350ec893e15a1298483f4a2" exitCode=0 Sep 10 20:18:31.627249 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:31.626867 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" event={"ID":"f53bd00c-7317-49be-a7e1-f31cb6401abb","Type":"ContainerDied","Data":"31d34ee49b9756786d4b60800041852c23bbe0395350ec893e15a1298483f4a2"} Sep 10 20:18:31.627249 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:31.626920 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" event={"ID":"f53bd00c-7317-49be-a7e1-f31cb6401abb","Type":"ContainerStarted","Data":"d7f8552df5698ae381d5b14f186c1aebe717eadc98dbad9e8010ffb8a2c5c6ee"} Sep 10 20:18:32.769346 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:32.769316 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:32.875313 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:32.875282 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53bd00c-7317-49be-a7e1-f31cb6401abb-maas-api-service-ca\") pod \"f53bd00c-7317-49be-a7e1-f31cb6401abb\" (UID: \"f53bd00c-7317-49be-a7e1-f31cb6401abb\") " Sep 10 20:18:32.875646 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:32.875623 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f53bd00c-7317-49be-a7e1-f31cb6401abb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f53bd00c-7317-49be-a7e1-f31cb6401abb" (UID: "f53bd00c-7317-49be-a7e1-f31cb6401abb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:32.976061 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:32.975982 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f53bd00c-7317-49be-a7e1-f31cb6401abb-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:33.640576 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:33.640543 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" event={"ID":"f53bd00c-7317-49be-a7e1-f31cb6401abb","Type":"ContainerDied","Data":"d7f8552df5698ae381d5b14f186c1aebe717eadc98dbad9e8010ffb8a2c5c6ee"} Sep 10 20:18:33.640576 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:33.640572 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t" Sep 10 20:18:33.640859 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:33.640598 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7f8552df5698ae381d5b14f186c1aebe717eadc98dbad9e8010ffb8a2c5c6ee" Sep 10 20:18:40.673934 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.673895 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk"] Sep 10 20:18:40.678687 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.678658 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-214b09-29817855-tjqbk"] Sep 10 20:18:40.706379 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.706341 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f"] Sep 10 20:18:40.706615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.706586 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" podUID="9edc4159-b99e-4ca4-bb52-bf2401855dbf" containerName="payload-processing" containerID="cri-o://d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6" gracePeriod=30 Sep 10 20:18:40.715810 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.715767 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj"] Sep 10 20:18:40.716136 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.716060 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" podUID="c62f98e4-92b8-4ad1-bc0a-feec991b2685" containerName="payload-pre-processing" containerID="cri-o://74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe" gracePeriod=30 Sep 10 20:18:40.875863 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.875829 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t"] Sep 10 20:18:40.885106 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.881985 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-214b09-q5n5t"] Sep 10 20:18:40.983610 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:40.983587 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:18:41.000665 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.000642 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:18:41.056353 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.056317 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx6lf\" (UniqueName: \"kubernetes.io/projected/9edc4159-b99e-4ca4-bb52-bf2401855dbf-kube-api-access-kx6lf\") pod \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " Sep 10 20:18:41.056758 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.056734 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c62f98e4-92b8-4ad1-bc0a-feec991b2685-plugins-config-volume\") pod \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " Sep 10 20:18:41.057042 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.057022 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9edc4159-b99e-4ca4-bb52-bf2401855dbf-plugins-config-volume\") pod \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\" (UID: \"9edc4159-b99e-4ca4-bb52-bf2401855dbf\") " Sep 10 20:18:41.057636 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.057606 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4bk8\" (UniqueName: \"kubernetes.io/projected/c62f98e4-92b8-4ad1-bc0a-feec991b2685-kube-api-access-s4bk8\") pod \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\" (UID: \"c62f98e4-92b8-4ad1-bc0a-feec991b2685\") " Sep 10 20:18:41.057750 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.057197 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c62f98e4-92b8-4ad1-bc0a-feec991b2685-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c62f98e4-92b8-4ad1-bc0a-feec991b2685" (UID: "c62f98e4-92b8-4ad1-bc0a-feec991b2685"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:41.057750 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.057545 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9edc4159-b99e-4ca4-bb52-bf2401855dbf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9edc4159-b99e-4ca4-bb52-bf2401855dbf" (UID: "9edc4159-b99e-4ca4-bb52-bf2401855dbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:41.057977 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.057958 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9edc4159-b99e-4ca4-bb52-bf2401855dbf-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:41.058135 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.058121 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c62f98e4-92b8-4ad1-bc0a-feec991b2685-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:41.059269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.059240 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9edc4159-b99e-4ca4-bb52-bf2401855dbf-kube-api-access-kx6lf" (OuterVolumeSpecName: "kube-api-access-kx6lf") pod "9edc4159-b99e-4ca4-bb52-bf2401855dbf" (UID: "9edc4159-b99e-4ca4-bb52-bf2401855dbf"). InnerVolumeSpecName "kube-api-access-kx6lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:18:41.060174 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.060145 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c62f98e4-92b8-4ad1-bc0a-feec991b2685-kube-api-access-s4bk8" (OuterVolumeSpecName: "kube-api-access-s4bk8") pod "c62f98e4-92b8-4ad1-bc0a-feec991b2685" (UID: "c62f98e4-92b8-4ad1-bc0a-feec991b2685"). InnerVolumeSpecName "kube-api-access-s4bk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:18:41.159307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.159273 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx6lf\" (UniqueName: \"kubernetes.io/projected/9edc4159-b99e-4ca4-bb52-bf2401855dbf-kube-api-access-kx6lf\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:41.159307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.159301 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4bk8\" (UniqueName: \"kubernetes.io/projected/c62f98e4-92b8-4ad1-bc0a-feec991b2685-kube-api-access-s4bk8\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:41.690683 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.690643 2570 generic.go:356] "Generic (PLEG): container finished" podID="9edc4159-b99e-4ca4-bb52-bf2401855dbf" containerID="d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6" exitCode=0 Sep 10 20:18:41.691195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.690690 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" Sep 10 20:18:41.691195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.690730 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" event={"ID":"9edc4159-b99e-4ca4-bb52-bf2401855dbf","Type":"ContainerDied","Data":"d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6"} Sep 10 20:18:41.691195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.690832 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f" event={"ID":"9edc4159-b99e-4ca4-bb52-bf2401855dbf","Type":"ContainerDied","Data":"ebb80053f4bd388d4c6c595d9314b197955ea9b7c4b7dab89a76f7ac0611fd2b"} Sep 10 20:18:41.691195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.690865 2570 scope.go:117] "RemoveContainer" containerID="d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6" Sep 10 20:18:41.692376 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.692353 2570 generic.go:356] "Generic (PLEG): container finished" podID="c62f98e4-92b8-4ad1-bc0a-feec991b2685" containerID="74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe" exitCode=0 Sep 10 20:18:41.692491 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.692443 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" event={"ID":"c62f98e4-92b8-4ad1-bc0a-feec991b2685","Type":"ContainerDied","Data":"74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe"} Sep 10 20:18:41.692491 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.692482 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" event={"ID":"c62f98e4-92b8-4ad1-bc0a-feec991b2685","Type":"ContainerDied","Data":"a135bb4febee55dabe57599ce411b1f6383a4b8770697486095282970b088262"} Sep 10 20:18:41.692601 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.692541 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj" Sep 10 20:18:41.703468 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.703446 2570 scope.go:117] "RemoveContainer" containerID="d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6" Sep 10 20:18:41.704332 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:18:41.704270 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6\": container with ID starting with d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6 not found: ID does not exist" containerID="d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6" Sep 10 20:18:41.704332 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.704309 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6"} err="failed to get container status \"d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6\": rpc error: code = NotFound desc = could not find container \"d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6\": container with ID starting with d12ac95d469d280359a3778289f3c7a0aa0c2f98db223aace3c7126329517ec6 not found: ID does not exist" Sep 10 20:18:41.704476 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.704332 2570 scope.go:117] "RemoveContainer" containerID="74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe" Sep 10 20:18:41.713359 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.713164 2570 scope.go:117] "RemoveContainer" containerID="74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe" Sep 10 20:18:41.713511 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:18:41.713492 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe\": container with ID starting with 74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe not found: ID does not exist" containerID="74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe" Sep 10 20:18:41.713574 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.713528 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe"} err="failed to get container status \"74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe\": rpc error: code = NotFound desc = could not find container \"74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe\": container with ID starting with 74c6c13a8785d487c864e66c8696e4c1a8f485d8df4958135e3eeec0f0f92cbe not found: ID does not exist" Sep 10 20:18:41.715978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.715950 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f"] Sep 10 20:18:41.718036 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.718018 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-214b09-f54bfdfc7-jgs8f"] Sep 10 20:18:41.726884 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.726864 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj"] Sep 10 20:18:41.734282 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.734263 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-214b09-bdd8869c4-7tvbj"] Sep 10 20:18:41.766260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.766237 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9edc4159-b99e-4ca4-bb52-bf2401855dbf" path="/var/lib/kubelet/pods/9edc4159-b99e-4ca4-bb52-bf2401855dbf/volumes" Sep 10 20:18:41.766587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.766575 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c62f98e4-92b8-4ad1-bc0a-feec991b2685" path="/var/lib/kubelet/pods/c62f98e4-92b8-4ad1-bc0a-feec991b2685/volumes" Sep 10 20:18:41.766823 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.766812 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f28253cb-62f3-4abc-ab17-0cedc96e9c45" path="/var/lib/kubelet/pods/f28253cb-62f3-4abc-ab17-0cedc96e9c45/volumes" Sep 10 20:18:41.767064 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:41.767053 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f53bd00c-7317-49be-a7e1-f31cb6401abb" path="/var/lib/kubelet/pods/f53bd00c-7317-49be-a7e1-f31cb6401abb/volumes" Sep 10 20:18:47.708672 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.708635 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4"] Sep 10 20:18:47.709118 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709104 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9edc4159-b99e-4ca4-bb52-bf2401855dbf" containerName="payload-processing" Sep 10 20:18:47.709118 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709119 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9edc4159-b99e-4ca4-bb52-bf2401855dbf" containerName="payload-processing" Sep 10 20:18:47.709213 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709132 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c62f98e4-92b8-4ad1-bc0a-feec991b2685" containerName="payload-pre-processing" Sep 10 20:18:47.709213 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709137 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c62f98e4-92b8-4ad1-bc0a-feec991b2685" containerName="payload-pre-processing" Sep 10 20:18:47.709213 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709146 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53bd00c-7317-49be-a7e1-f31cb6401abb" containerName="revoke-keys" Sep 10 20:18:47.709213 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709151 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53bd00c-7317-49be-a7e1-f31cb6401abb" containerName="revoke-keys" Sep 10 20:18:47.709336 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709216 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53bd00c-7317-49be-a7e1-f31cb6401abb" containerName="revoke-keys" Sep 10 20:18:47.709336 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709227 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c62f98e4-92b8-4ad1-bc0a-feec991b2685" containerName="payload-pre-processing" Sep 10 20:18:47.709336 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.709233 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9edc4159-b99e-4ca4-bb52-bf2401855dbf" containerName="payload-processing" Sep 10 20:18:47.712316 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.712300 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:47.715367 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.715340 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:18:47.715367 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.715347 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:18:47.723681 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.723656 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4"] Sep 10 20:18:47.827649 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.827610 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7cf3f-bb0e-4a00-9087-79d748e26161-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4\" (UID: \"cfd7cf3f-bb0e-4a00-9087-79d748e26161\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:47.928219 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.928189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7cf3f-bb0e-4a00-9087-79d748e26161-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4\" (UID: \"cfd7cf3f-bb0e-4a00-9087-79d748e26161\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:47.928870 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:47.928850 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7cf3f-bb0e-4a00-9087-79d748e26161-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4\" (UID: \"cfd7cf3f-bb0e-4a00-9087-79d748e26161\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:48.023200 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:48.023114 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:48.298599 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:48.298568 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4"] Sep 10 20:18:48.299903 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:18:48.299876 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd7cf3f_bb0e_4a00_9087_79d748e26161.slice/crio-e19cfef6a2a8f71f942d921012d9610aee9746a15641c0dd4c50a961d158b119 WatchSource:0}: Error finding container e19cfef6a2a8f71f942d921012d9610aee9746a15641c0dd4c50a961d158b119: Status 404 returned error can't find the container with id e19cfef6a2a8f71f942d921012d9610aee9746a15641c0dd4c50a961d158b119 Sep 10 20:18:48.743959 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:48.743870 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" event={"ID":"cfd7cf3f-bb0e-4a00-9087-79d748e26161","Type":"ContainerStarted","Data":"e19cfef6a2a8f71f942d921012d9610aee9746a15641c0dd4c50a961d158b119"} Sep 10 20:18:49.444137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.444040 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm"] Sep 10 20:18:49.446877 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.446859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.449450 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.449424 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-316fde-openshift-default-dockercfg-5tvq6\"" Sep 10 20:18:49.459536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.459513 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm"] Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550610 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-podinfo\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550668 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-envoy\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550701 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istiod-ca-cert\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550734 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-socket\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550776 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-token\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550817 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-credential-socket\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550907 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-certs\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.550934 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp6lk\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-kube-api-access-lp6lk\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.551176 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.551007 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-data\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652258 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652221 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-token\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652258 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652265 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-credential-socket\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-certs\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652310 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6lk\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-kube-api-access-lp6lk\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-data\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-podinfo\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652495 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-envoy\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652777 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652523 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istiod-ca-cert\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652777 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652665 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-certs\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652777 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-socket\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652777 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652746 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-credential-socket\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652811 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-data\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.652994 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.652950 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-socket\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.653146 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.653123 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istiod-ca-cert\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.654689 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.654664 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-envoy\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.655144 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.655121 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-podinfo\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.660112 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.660088 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-token\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.660736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.660719 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6lk\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-kube-api-access-lp6lk\") pod \"e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:49.751488 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.751450 2570 generic.go:356] "Generic (PLEG): container finished" podID="cfd7cf3f-bb0e-4a00-9087-79d748e26161" containerID="6f7ad47193fdf234680ac4ad13f768c979db3f3f537be8ee7fabb559ca862ba7" exitCode=0 Sep 10 20:18:49.751882 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.751499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" event={"ID":"cfd7cf3f-bb0e-4a00-9087-79d748e26161","Type":"ContainerDied","Data":"6f7ad47193fdf234680ac4ad13f768c979db3f3f537be8ee7fabb559ca862ba7"} Sep 10 20:18:49.760421 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:49.760397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:50.051145 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.051120 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm"] Sep 10 20:18:50.053873 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:18:50.053837 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432a515f_31a2_4ca1_9d3e_d1624bbdced7.slice/crio-fb16ea62a5b8c2f8b20423bfe2054e163999c6164f6765ab535f0920a6f6f7f5 WatchSource:0}: Error finding container fb16ea62a5b8c2f8b20423bfe2054e163999c6164f6765ab535f0920a6f6f7f5: Status 404 returned error can't find the container with id fb16ea62a5b8c2f8b20423bfe2054e163999c6164f6765ab535f0920a6f6f7f5 Sep 10 20:18:50.056204 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.056171 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:18:50.056271 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.056257 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:18:50.056306 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.056286 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:18:50.759136 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.759097 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" event={"ID":"432a515f-31a2-4ca1-9d3e-d1624bbdced7","Type":"ContainerStarted","Data":"d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624"} Sep 10 20:18:50.759655 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.759175 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" event={"ID":"432a515f-31a2-4ca1-9d3e-d1624bbdced7","Type":"ContainerStarted","Data":"fb16ea62a5b8c2f8b20423bfe2054e163999c6164f6765ab535f0920a6f6f7f5"} Sep 10 20:18:50.760968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.760945 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:50.779673 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.779617 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podStartSLOduration=1.779596854 podStartE2EDuration="1.779596854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:18:50.778312629 +0000 UTC m=+1499.499259816" watchObservedRunningTime="2026-09-10 20:18:50.779596854 +0000 UTC m=+1499.500543974" Sep 10 20:18:50.903844 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:50.903807 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:51.068887 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.068801 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7cf3f-bb0e-4a00-9087-79d748e26161-maas-api-service-ca\") pod \"cfd7cf3f-bb0e-4a00-9087-79d748e26161\" (UID: \"cfd7cf3f-bb0e-4a00-9087-79d748e26161\") " Sep 10 20:18:51.069232 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.069207 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd7cf3f-bb0e-4a00-9087-79d748e26161-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cfd7cf3f-bb0e-4a00-9087-79d748e26161" (UID: "cfd7cf3f-bb0e-4a00-9087-79d748e26161"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:18:51.170351 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.170314 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd7cf3f-bb0e-4a00-9087-79d748e26161-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:18:51.767338 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.767308 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" Sep 10 20:18:51.767817 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.767629 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:51.767817 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.767659 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4" event={"ID":"cfd7cf3f-bb0e-4a00-9087-79d748e26161","Type":"ContainerDied","Data":"e19cfef6a2a8f71f942d921012d9610aee9746a15641c0dd4c50a961d158b119"} Sep 10 20:18:51.767817 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:51.767684 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e19cfef6a2a8f71f942d921012d9610aee9746a15641c0dd4c50a961d158b119" Sep 10 20:18:52.772659 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:52.772630 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:52.773785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:52.773764 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:18:58.641949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:58.641906 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4"] Sep 10 20:18:58.645531 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:58.645499 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-6ef723-5v6g4"] Sep 10 20:18:59.765921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:18:59.765886 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd7cf3f-bb0e-4a00-9087-79d748e26161" path="/var/lib/kubelet/pods/cfd7cf3f-bb0e-4a00-9087-79d748e26161/volumes" Sep 10 20:19:05.002487 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:05.002455 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr"] Sep 10 20:19:05.002931 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:05.002693 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" containerID="cri-o://cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b" gracePeriod=30 Sep 10 20:19:08.815283 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:08.815238 2570 patch_prober.go:28] interesting pod/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 20:19:08.815764 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:08.815322 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 20:19:10.262260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.262235 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:19:10.366482 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366402 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lfgc\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-kube-api-access-5lfgc\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366482 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366464 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-socket\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366698 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366493 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-certs\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366698 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366524 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22e13592-856a-43cf-8076-4b8edb7cb84e-istiod-ca-cert\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366698 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366552 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-credential-socket\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366698 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366665 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-podinfo\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366908 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366706 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-data\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366908 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366728 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-token\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366908 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366755 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-envoy\") pod \"22e13592-856a-43cf-8076-4b8edb7cb84e\" (UID: \"22e13592-856a-43cf-8076-4b8edb7cb84e\") " Sep 10 20:19:10.366908 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366838 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:19:10.366908 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366875 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:19:10.366908 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.366891 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22e13592-856a-43cf-8076-4b8edb7cb84e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:19:10.367250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.367026 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-data" (OuterVolumeSpecName: "istio-data") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:19:10.367250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.367169 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.367250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.367185 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.367250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.367197 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.367250 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.367209 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22e13592-856a-43cf-8076-4b8edb7cb84e-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.367490 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.367355 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:19:10.368968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.368941 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-kube-api-access-5lfgc" (OuterVolumeSpecName: "kube-api-access-5lfgc") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "kube-api-access-5lfgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:19:10.369284 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.369262 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:19:10.369405 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.369386 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-token" (OuterVolumeSpecName: "istio-token") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:19:10.369544 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.369517 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22e13592-856a-43cf-8076-4b8edb7cb84e" (UID: "22e13592-856a-43cf-8076-4b8edb7cb84e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:19:10.468025 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.467992 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.468025 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.468021 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.468025 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.468030 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.468269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.468038 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lfgc\" (UniqueName: \"kubernetes.io/projected/22e13592-856a-43cf-8076-4b8edb7cb84e-kube-api-access-5lfgc\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.468269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.468049 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22e13592-856a-43cf-8076-4b8edb7cb84e-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:19:10.891091 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.890950 2570 generic.go:356] "Generic (PLEG): container finished" podID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerID="cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b" exitCode=0 Sep 10 20:19:10.891091 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.890997 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" Sep 10 20:19:10.891091 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.891045 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" event={"ID":"22e13592-856a-43cf-8076-4b8edb7cb84e","Type":"ContainerDied","Data":"cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b"} Sep 10 20:19:10.891444 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.891123 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr" event={"ID":"22e13592-856a-43cf-8076-4b8edb7cb84e","Type":"ContainerDied","Data":"bdab09058d032c639a1d56ac299999511a372a0f1c280c25ac32cc8437a971f4"} Sep 10 20:19:10.891444 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.891149 2570 scope.go:117] "RemoveContainer" containerID="cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b" Sep 10 20:19:10.892816 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.892790 2570 generic.go:356] "Generic (PLEG): container finished" podID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerID="d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624" exitCode=0 Sep 10 20:19:10.892928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.892859 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" event={"ID":"432a515f-31a2-4ca1-9d3e-d1624bbdced7","Type":"ContainerDied","Data":"d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624"} Sep 10 20:19:10.893228 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.893215 2570 scope.go:117] "RemoveContainer" containerID="d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624" Sep 10 20:19:10.894276 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.894245 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:10.894388 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.894322 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:10.894388 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.894356 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:10.901593 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.901580 2570 scope.go:117] "RemoveContainer" containerID="603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a" Sep 10 20:19:10.910146 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.910128 2570 scope.go:117] "RemoveContainer" containerID="cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b" Sep 10 20:19:10.910413 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:19:10.910393 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b\": container with ID starting with cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b not found: ID does not exist" containerID="cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b" Sep 10 20:19:10.910507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.910423 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b"} err="failed to get container status \"cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b\": rpc error: code = NotFound desc = could not find container \"cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b\": container with ID starting with cce07bddddd6041837029bf74ac815132fa0325ee4645e9a2c81acc837ff384b not found: ID does not exist" Sep 10 20:19:10.910507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.910439 2570 scope.go:117] "RemoveContainer" containerID="603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a" Sep 10 20:19:10.910683 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:19:10.910669 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a\": container with ID starting with 603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a not found: ID does not exist" containerID="603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a" Sep 10 20:19:10.910729 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.910686 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a"} err="failed to get container status \"603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a\": rpc error: code = NotFound desc = could not find container \"603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a\": container with ID starting with 603df8114fbeef64be9861ce9fc9657c421282a09304092f219ade8f325e705a not found: ID does not exist" Sep 10 20:19:10.938490 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.938458 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr"] Sep 10 20:19:10.942183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:10.942143 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6ef723-openshift-default-65486dc578-9xqbr"] Sep 10 20:19:11.769981 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:11.769947 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" path="/var/lib/kubelet/pods/22e13592-856a-43cf-8076-4b8edb7cb84e/volumes" Sep 10 20:19:11.901369 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:11.901333 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" event={"ID":"432a515f-31a2-4ca1-9d3e-d1624bbdced7","Type":"ContainerStarted","Data":"c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0"} Sep 10 20:19:12.760827 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:12.760783 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:19:12.761057 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:12.760851 2570 patch_prober.go:28] interesting pod/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Sep 10 20:19:12.761057 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:12.760924 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Sep 10 20:19:13.763141 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:13.763101 2570 patch_prober.go:28] interesting pod/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Sep 10 20:19:13.763664 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:13.763192 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Sep 10 20:19:14.761706 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:14.761668 2570 patch_prober.go:28] interesting pod/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Sep 10 20:19:14.761991 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:14.761964 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Sep 10 20:19:15.760939 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:15.760898 2570 patch_prober.go:28] interesting pod/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Sep 10 20:19:15.761584 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:15.761552 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Sep 10 20:19:16.760849 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:16.760814 2570 patch_prober.go:28] interesting pod/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Sep 10 20:19:16.761058 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:16.760873 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Sep 10 20:19:17.765991 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:17.765963 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:19:17.766427 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:17.766269 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:19:17.766894 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:17.766876 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:19:18.556590 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.554781 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp"] Sep 10 20:19:18.560825 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560789 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" Sep 10 20:19:18.560825 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560822 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560866 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd7cf3f-bb0e-4a00-9087-79d748e26161" containerName="revoke-keys" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560875 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd7cf3f-bb0e-4a00-9087-79d748e26161" containerName="revoke-keys" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560888 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560903 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.560991 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd7cf3f-bb0e-4a00-9087-79d748e26161" containerName="revoke-keys" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.561006 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" Sep 10 20:19:18.561063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.561015 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e13592-856a-43cf-8076-4b8edb7cb84e" containerName="istio-proxy" Sep 10 20:19:18.565206 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.565178 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.565659 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.565633 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp"] Sep 10 20:19:18.567599 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.567578 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-82457a-openshift-default-dockercfg-f87tv\"" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653313 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-credential-socket\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653367 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-data\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-certs\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-envoy\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653536 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-token\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653583 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-podinfo\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-socket\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653645 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlqm7\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-kube-api-access-wlqm7\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.656484 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.653691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istiod-ca-cert\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.754701 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.754661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-envoy\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.754953 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.754720 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-token\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.754953 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.754752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-podinfo\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.754953 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.754776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-socket\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.754953 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.754794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlqm7\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-kube-api-access-wlqm7\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755217 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.754961 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istiod-ca-cert\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755217 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-credential-socket\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755217 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755057 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-data\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755217 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755145 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-certs\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755217 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755183 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-socket\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755482 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755405 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-credential-socket\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755661 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755641 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-certs\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755669 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-data\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.755775 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.755729 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istiod-ca-cert\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.756998 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.756975 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-envoy\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.757297 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.757276 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-podinfo\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.763494 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.763473 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-token\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.763716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.763695 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlqm7\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-kube-api-access-wlqm7\") pod \"e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:18.878245 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:18.878153 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:19.160803 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.160770 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp"] Sep 10 20:19:19.163777 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:19:19.163743 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9c0a83_65c4_4ccc_bffd_01e0318249f8.slice/crio-91d19a54993bddf9ddf951eb971325293e8405163697deeb0f9b0777b5dd0ba5 WatchSource:0}: Error finding container 91d19a54993bddf9ddf951eb971325293e8405163697deeb0f9b0777b5dd0ba5: Status 404 returned error can't find the container with id 91d19a54993bddf9ddf951eb971325293e8405163697deeb0f9b0777b5dd0ba5 Sep 10 20:19:19.166106 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.166053 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:19.166186 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.166143 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:19.166186 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.166178 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:19.953028 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.952991 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" event={"ID":"8b9c0a83-65c4-4ccc-bffd-01e0318249f8","Type":"ContainerStarted","Data":"b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8"} Sep 10 20:19:19.953440 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.953047 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" event={"ID":"8b9c0a83-65c4-4ccc-bffd-01e0318249f8","Type":"ContainerStarted","Data":"91d19a54993bddf9ddf951eb971325293e8405163697deeb0f9b0777b5dd0ba5"} Sep 10 20:19:19.979586 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:19.979536 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podStartSLOduration=1.979521154 podStartE2EDuration="1.979521154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:19:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:19:19.977381243 +0000 UTC m=+1528.698328375" watchObservedRunningTime="2026-09-10 20:19:19.979521154 +0000 UTC m=+1528.700468272" Sep 10 20:19:20.879218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:20.879187 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:20.884157 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:20.884134 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:20.959581 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:20.959547 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:20.960626 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:20.960607 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:23.606432 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.603297 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c"] Sep 10 20:19:23.608913 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.608879 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.611300 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.611277 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-82457a\"" Sep 10 20:19:23.611619 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.611602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-82457a-dockercfg-5cz69\"" Sep 10 20:19:23.616453 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.616428 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c"] Sep 10 20:19:23.711377 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.710665 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx5n7\" (UniqueName: \"kubernetes.io/projected/3b424483-1c6f-495f-b9a6-d882a9b30bea-kube-api-access-zx5n7\") pod \"payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.711377 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.710769 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b424483-1c6f-495f-b9a6-d882a9b30bea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.811564 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.811533 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx5n7\" (UniqueName: \"kubernetes.io/projected/3b424483-1c6f-495f-b9a6-d882a9b30bea-kube-api-access-zx5n7\") pod \"payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.811739 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.811603 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b424483-1c6f-495f-b9a6-d882a9b30bea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.812400 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.812380 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b424483-1c6f-495f-b9a6-d882a9b30bea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.819764 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.819738 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx5n7\" (UniqueName: \"kubernetes.io/projected/3b424483-1c6f-495f-b9a6-d882a9b30bea-kube-api-access-zx5n7\") pod \"payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:23.923528 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:23.923450 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:24.204493 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:24.204457 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c"] Sep 10 20:19:24.205530 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:19:24.205499 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b424483_1c6f_495f_b9a6_d882a9b30bea.slice/crio-454e3eb96c5fe87978d4718150605d6e1fe43e3a6572472688aa3d42d5683c20 WatchSource:0}: Error finding container 454e3eb96c5fe87978d4718150605d6e1fe43e3a6572472688aa3d42d5683c20: Status 404 returned error can't find the container with id 454e3eb96c5fe87978d4718150605d6e1fe43e3a6572472688aa3d42d5683c20 Sep 10 20:19:24.987171 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:24.987134 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" event={"ID":"3b424483-1c6f-495f-b9a6-d882a9b30bea","Type":"ContainerStarted","Data":"bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154"} Sep 10 20:19:24.987568 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:24.987190 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" event={"ID":"3b424483-1c6f-495f-b9a6-d882a9b30bea","Type":"ContainerStarted","Data":"454e3eb96c5fe87978d4718150605d6e1fe43e3a6572472688aa3d42d5683c20"} Sep 10 20:19:24.987568 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:24.987216 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:25.004592 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:25.004543 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" podStartSLOduration=2.004529212 podStartE2EDuration="2.004529212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:19:25.002978979 +0000 UTC m=+1533.723926097" watchObservedRunningTime="2026-09-10 20:19:25.004529212 +0000 UTC m=+1533.725476330" Sep 10 20:19:35.994268 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:35.994232 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:19:40.095528 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:40.095498 2570 generic.go:356] "Generic (PLEG): container finished" podID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerID="b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8" exitCode=0 Sep 10 20:19:40.096031 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:40.095575 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" event={"ID":"8b9c0a83-65c4-4ccc-bffd-01e0318249f8","Type":"ContainerDied","Data":"b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8"} Sep 10 20:19:40.096031 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:40.095953 2570 scope.go:117] "RemoveContainer" containerID="b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8" Sep 10 20:19:40.097040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:40.097010 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:40.097116 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:40.097103 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:40.097158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:40.097138 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:19:41.103527 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:41.103489 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" event={"ID":"8b9c0a83-65c4-4ccc-bffd-01e0318249f8","Type":"ContainerStarted","Data":"76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14"} Sep 10 20:19:41.879141 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:41.879107 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:41.879328 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:41.879256 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Sep 10 20:19:41.879328 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:41.879305 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Sep 10 20:19:42.878659 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:42.878626 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Sep 10 20:19:42.879190 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:42.878679 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Sep 10 20:19:43.879367 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:43.879331 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Sep 10 20:19:43.879792 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:43.879390 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Sep 10 20:19:44.879265 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:44.879228 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Sep 10 20:19:44.879442 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:44.879294 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Sep 10 20:19:45.879284 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:45.879246 2570 patch_prober.go:28] interesting pod/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Sep 10 20:19:45.879468 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:45.879302 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Sep 10 20:19:46.882615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:46.882575 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:46.883120 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:46.883101 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:46.883862 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:46.883842 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:19:51.993285 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:19:51.993229 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6v58f_3e8f86ab-11fb-41d8-97e2-0b1077ea1e03/payload-processing/0.log" Sep 10 20:20:43.221082 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:43.221048 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6v58f_3e8f86ab-11fb-41d8-97e2-0b1077ea1e03/payload-processing/0.log" Sep 10 20:20:43.919965 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:43.919769 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv"] Sep 10 20:20:43.923813 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:43.923782 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:43.926565 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:43.926537 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-fe603d-openshift-default-dockercfg-qw8qj\"" Sep 10 20:20:43.938129 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:43.938056 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv"] Sep 10 20:20:44.019717 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.019679 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-envoy\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.020053 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.020026 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qljzv\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-kube-api-access-qljzv\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.020856 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.020834 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-credential-socket\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.021054 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.021024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-data\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.021211 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.021119 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.021211 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.021150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-socket\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.021211 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.021192 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-token\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.021393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.021230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-certs\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.021393 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.021299 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-podinfo\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.121994 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.121908 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-token\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.121994 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.121948 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-certs\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.121994 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.121987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-podinfo\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122027 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-envoy\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qljzv\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-kube-api-access-qljzv\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-credential-socket\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122165 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-data\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122248 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-socket\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122625 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122402 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-certs\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122625 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122547 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-socket\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122729 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122687 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-credential-socket\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.122786 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.122763 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-data\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.123028 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.123008 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.124568 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.124543 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-envoy\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.124697 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.124683 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-podinfo\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.130640 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.130615 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-token\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.130859 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.130841 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qljzv\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-kube-api-access-qljzv\") pod \"e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.238801 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.238719 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:44.514230 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.514201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv"] Sep 10 20:20:44.515868 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:20:44.515841 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4a2eb2_dee4_49f7_8989_bb3071d310c2.slice/crio-959be44863af62071d14a67b70d3dac1cd37484a1ab095102e4bc91819b620d1 WatchSource:0}: Error finding container 959be44863af62071d14a67b70d3dac1cd37484a1ab095102e4bc91819b620d1: Status 404 returned error can't find the container with id 959be44863af62071d14a67b70d3dac1cd37484a1ab095102e4bc91819b620d1 Sep 10 20:20:44.518224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.518195 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:20:44.518298 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.518261 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:20:44.518298 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:44.518290 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:20:45.466735 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:45.466696 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" event={"ID":"7b4a2eb2-dee4-49f7-8989-bb3071d310c2","Type":"ContainerStarted","Data":"5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b"} Sep 10 20:20:45.467205 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:45.466747 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" event={"ID":"7b4a2eb2-dee4-49f7-8989-bb3071d310c2","Type":"ContainerStarted","Data":"959be44863af62071d14a67b70d3dac1cd37484a1ab095102e4bc91819b620d1"} Sep 10 20:20:45.488206 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:45.488156 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podStartSLOduration=2.4881373829999998 podStartE2EDuration="2.488137383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:20:45.48672314 +0000 UTC m=+1614.207670256" watchObservedRunningTime="2026-09-10 20:20:45.488137383 +0000 UTC m=+1614.209084502" Sep 10 20:20:46.239662 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:46.239625 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:46.245456 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:46.245431 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:46.473402 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:46.473366 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:46.474291 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:46.474271 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:20:49.470325 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.470278 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk"] Sep 10 20:20:49.473812 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.473786 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.476429 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.476404 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-fe603d\"" Sep 10 20:20:49.476565 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.476407 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-fe603d-dockercfg-2w5b4\"" Sep 10 20:20:49.483520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.483498 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk"] Sep 10 20:20:49.581052 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.581009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.581052 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.581056 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8xmg\" (UniqueName: \"kubernetes.io/projected/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-kube-api-access-m8xmg\") pod \"payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.682986 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.682938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.683208 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.682995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8xmg\" (UniqueName: \"kubernetes.io/projected/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-kube-api-access-m8xmg\") pod \"payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.683922 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.683889 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.691928 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.691896 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8xmg\" (UniqueName: \"kubernetes.io/projected/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-kube-api-access-m8xmg\") pod \"payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:49.786441 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:49.786409 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:50.062835 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:50.062806 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk"] Sep 10 20:20:50.065237 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:20:50.065209 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6b82e7_805a_49a8_a0d3_d5f37a961fa1.slice/crio-c64c3d2f4189d6a5fa97c8329fdd5992a59c8cf27a39edb6dc7669d00585302a WatchSource:0}: Error finding container c64c3d2f4189d6a5fa97c8329fdd5992a59c8cf27a39edb6dc7669d00585302a: Status 404 returned error can't find the container with id c64c3d2f4189d6a5fa97c8329fdd5992a59c8cf27a39edb6dc7669d00585302a Sep 10 20:20:50.497823 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:50.497788 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" event={"ID":"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1","Type":"ContainerStarted","Data":"7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5"} Sep 10 20:20:50.498239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:50.497848 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" event={"ID":"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1","Type":"ContainerStarted","Data":"c64c3d2f4189d6a5fa97c8329fdd5992a59c8cf27a39edb6dc7669d00585302a"} Sep 10 20:20:50.498239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:50.497970 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:20:50.515364 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:20:50.515316 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" podStartSLOduration=1.5152991930000002 podStartE2EDuration="1.515299193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:20:50.513420984 +0000 UTC m=+1619.234368116" watchObservedRunningTime="2026-09-10 20:20:50.515299193 +0000 UTC m=+1619.236246345" Sep 10 20:21:01.506587 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:01.506555 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:21:07.614087 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:07.614043 2570 generic.go:356] "Generic (PLEG): container finished" podID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerID="5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b" exitCode=0 Sep 10 20:21:07.614502 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:07.614125 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" event={"ID":"7b4a2eb2-dee4-49f7-8989-bb3071d310c2","Type":"ContainerDied","Data":"5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b"} Sep 10 20:21:07.614557 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:07.614525 2570 scope.go:117] "RemoveContainer" containerID="5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b" Sep 10 20:21:07.615824 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:07.615792 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:21:07.615935 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:07.615899 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:21:07.616004 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:07.615950 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 20:21:08.622403 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:08.622370 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" event={"ID":"7b4a2eb2-dee4-49f7-8989-bb3071d310c2","Type":"ContainerStarted","Data":"d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40"} Sep 10 20:21:09.239171 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:09.239127 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:21:09.239524 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:09.239279 2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Sep 10 20:21:09.239738 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:09.239711 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Sep 10 20:21:10.239764 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:10.239718 2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Sep 10 20:21:10.240290 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:10.239789 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Sep 10 20:21:11.245094 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:11.240169 2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Sep 10 20:21:11.245094 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:11.240248 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Sep 10 20:21:12.244662 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:12.244619 2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Sep 10 20:21:12.244882 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:12.244697 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Sep 10 20:21:13.245094 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:13.241415 2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Sep 10 20:21:13.245094 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:13.241516 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Sep 10 20:21:14.243538 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:14.243507 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:21:14.243941 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:14.243918 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:21:14.244630 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:14.244614 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:21:19.319445 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.319404 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk"] Sep 10 20:21:19.319932 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.319653 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" podUID="3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" containerName="payload-processing" containerID="cri-o://7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5" gracePeriod=30 Sep 10 20:21:19.574474 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.574415 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:21:19.695260 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.695225 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8xmg\" (UniqueName: \"kubernetes.io/projected/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-kube-api-access-m8xmg\") pod \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " Sep 10 20:21:19.695435 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.695310 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-plugins-config-volume\") pod \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\" (UID: \"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1\") " Sep 10 20:21:19.695685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.695658 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" (UID: "3e6b82e7-805a-49a8-a0d3-d5f37a961fa1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:21:19.697421 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.697400 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-kube-api-access-m8xmg" (OuterVolumeSpecName: "kube-api-access-m8xmg") pod "3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" (UID: "3e6b82e7-805a-49a8-a0d3-d5f37a961fa1"). InnerVolumeSpecName "kube-api-access-m8xmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:21:19.705164 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.705133 2570 generic.go:356] "Generic (PLEG): container finished" podID="3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" containerID="7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5" exitCode=0 Sep 10 20:21:19.705300 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.705179 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" Sep 10 20:21:19.705300 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.705180 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" event={"ID":"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1","Type":"ContainerDied","Data":"7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5"} Sep 10 20:21:19.705300 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.705238 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk" event={"ID":"3e6b82e7-805a-49a8-a0d3-d5f37a961fa1","Type":"ContainerDied","Data":"c64c3d2f4189d6a5fa97c8329fdd5992a59c8cf27a39edb6dc7669d00585302a"} Sep 10 20:21:19.705300 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.705263 2570 scope.go:117] "RemoveContainer" containerID="7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5" Sep 10 20:21:19.715627 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.715611 2570 scope.go:117] "RemoveContainer" containerID="7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5" Sep 10 20:21:19.715899 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:21:19.715879 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5\": container with ID starting with 7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5 not found: ID does not exist" containerID="7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5" Sep 10 20:21:19.715949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.715907 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5"} err="failed to get container status \"7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5\": rpc error: code = NotFound desc = could not find container \"7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5\": container with ID starting with 7172364cf5bcc73fc23e092b4a665d1557de6027a9b85e4aa43e997382ae82b5 not found: ID does not exist" Sep 10 20:21:19.727418 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.727393 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk"] Sep 10 20:21:19.730790 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.730768 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fe603d-5d8f7c9fbc-9m5qk"] Sep 10 20:21:19.766025 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.766000 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" path="/var/lib/kubelet/pods/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1/volumes" Sep 10 20:21:19.796331 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.796302 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8xmg\" (UniqueName: \"kubernetes.io/projected/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-kube-api-access-m8xmg\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:19.796331 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:19.796327 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e6b82e7-805a-49a8-a0d3-d5f37a961fa1-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:25.772115 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:25.772080 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv"] Sep 10 20:21:25.772605 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:25.772326 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" containerID="cri-o://d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40" gracePeriod=30 Sep 10 20:21:29.246123 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:29.246081 2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 20:21:29.246499 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:29.246146 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 20:21:31.025338 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.025314 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:21:31.103386 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103312 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-credential-socket\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103386 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103345 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istiod-ca-cert\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103386 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103368 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-data\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103391 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-certs\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103412 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-token\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103436 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-envoy\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103465 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qljzv\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-kube-api-access-qljzv\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103506 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-socket\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103657 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103607 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:21:31.103947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103710 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-podinfo\") pod \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\" (UID: \"7b4a2eb2-dee4-49f7-8989-bb3071d310c2\") " Sep 10 20:21:31.103947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103726 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:21:31.103947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103746 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:21:31.103947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103787 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:21:31.103947 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.103860 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:21:31.104218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.104168 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.104218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.104188 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.104218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.104203 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.104218 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.104217 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.104374 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.104230 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.105733 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.105702 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:21:31.105844 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.105751 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:21:31.106236 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.106220 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:21:31.106287 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.106234 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-kube-api-access-qljzv" (OuterVolumeSpecName: "kube-api-access-qljzv") pod "7b4a2eb2-dee4-49f7-8989-bb3071d310c2" (UID: "7b4a2eb2-dee4-49f7-8989-bb3071d310c2"). InnerVolumeSpecName "kube-api-access-qljzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:21:31.205062 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.205028 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.205062 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.205054 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.205062 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.205063 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.205062 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.205091 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qljzv\" (UniqueName: \"kubernetes.io/projected/7b4a2eb2-dee4-49f7-8989-bb3071d310c2-kube-api-access-qljzv\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:21:31.776453 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.776422 2570 generic.go:356] "Generic (PLEG): container finished" podID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerID="d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40" exitCode=0 Sep 10 20:21:31.776539 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.776471 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" Sep 10 20:21:31.776539 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.776497 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" event={"ID":"7b4a2eb2-dee4-49f7-8989-bb3071d310c2","Type":"ContainerDied","Data":"d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40"} Sep 10 20:21:31.776611 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.776562 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv" event={"ID":"7b4a2eb2-dee4-49f7-8989-bb3071d310c2","Type":"ContainerDied","Data":"959be44863af62071d14a67b70d3dac1cd37484a1ab095102e4bc91819b620d1"} Sep 10 20:21:31.776611 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.776585 2570 scope.go:117] "RemoveContainer" containerID="d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40" Sep 10 20:21:31.786391 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.786368 2570 scope.go:117] "RemoveContainer" containerID="5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b" Sep 10 20:21:31.795328 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.795303 2570 scope.go:117] "RemoveContainer" containerID="d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40" Sep 10 20:21:31.795646 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:21:31.795619 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40\": container with ID starting with d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40 not found: ID does not exist" containerID="d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40" Sep 10 20:21:31.795712 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.795647 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40"} err="failed to get container status \"d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40\": rpc error: code = NotFound desc = could not find container \"d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40\": container with ID starting with d7364e16be48a99f6a3873cf1c76153bfb042547797751a17755c2600bd5ea40 not found: ID does not exist" Sep 10 20:21:31.795712 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.795665 2570 scope.go:117] "RemoveContainer" containerID="5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b" Sep 10 20:21:31.795968 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:21:31.795948 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b\": container with ID starting with 5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b not found: ID does not exist" containerID="5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b" Sep 10 20:21:31.796017 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.795974 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b"} err="failed to get container status \"5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b\": rpc error: code = NotFound desc = could not find container \"5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b\": container with ID starting with 5c53b2bfe91c5acf16ee212905e5ad63725cc0d3479596f0c7d5ea148a6fcd3b not found: ID does not exist" Sep 10 20:21:31.797147 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.796930 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv"] Sep 10 20:21:31.799402 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:31.799383 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fe603d-openshift-default-c9d55497b-xsngv"] Sep 10 20:21:33.772496 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:33.772464 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" path="/var/lib/kubelet/pods/7b4a2eb2-dee4-49f7-8989-bb3071d310c2/volumes" Sep 10 20:21:52.041375 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:52.041342 2570 scope.go:117] "RemoveContainer" containerID="9b8243cfa368b48bde4f4b7f47ba4096ae3de1cd6d7a79f46c7663478e5832bf" Sep 10 20:21:52.050580 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:21:52.050559 2570 scope.go:117] "RemoveContainer" containerID="fa1357c787a002c34724f82e0d56abf4575bdc44b0e79289955939948c17c1cc" Sep 10 20:22:33.411601 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:33.411564 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm"] Sep 10 20:22:33.412142 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:33.411813 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" containerID="cri-o://c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0" gracePeriod=30 Sep 10 20:22:38.664137 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.664110 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:22:38.683233 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683159 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-envoy\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683233 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683199 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-podinfo\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683233 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683237 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istiod-ca-cert\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683273 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-credential-socket\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683292 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-socket\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683338 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-data\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683366 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-certs\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683438 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-token\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683556 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683473 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp6lk\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-kube-api-access-lp6lk\") pod \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\" (UID: \"432a515f-31a2-4ca1-9d3e-d1624bbdced7\") " Sep 10 20:22:38.683865 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683578 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:38.683865 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683594 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:38.683865 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683645 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-data" (OuterVolumeSpecName: "istio-data") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:38.683865 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683720 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:38.683865 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683758 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:22:38.684143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683907 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.684143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683927 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.684143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683942 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.684143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683956 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.684143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.683969 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.685531 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.685511 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-kube-api-access-lp6lk" (OuterVolumeSpecName: "kube-api-access-lp6lk") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "kube-api-access-lp6lk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:22:38.685647 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.685630 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-token" (OuterVolumeSpecName: "istio-token") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:22:38.685812 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.685782 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:38.685925 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.685826 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "432a515f-31a2-4ca1-9d3e-d1624bbdced7" (UID: "432a515f-31a2-4ca1-9d3e-d1624bbdced7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:22:38.784665 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.784623 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.784665 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.784652 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp6lk\" (UniqueName: \"kubernetes.io/projected/432a515f-31a2-4ca1-9d3e-d1624bbdced7-kube-api-access-lp6lk\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.784665 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.784666 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:38.784999 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:38.784677 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432a515f-31a2-4ca1-9d3e-d1624bbdced7-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:39.152066 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.152033 2570 generic.go:356] "Generic (PLEG): container finished" podID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerID="c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0" exitCode=0 Sep 10 20:22:39.152269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.152092 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" Sep 10 20:22:39.152269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.152117 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" event={"ID":"432a515f-31a2-4ca1-9d3e-d1624bbdced7","Type":"ContainerDied","Data":"c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0"} Sep 10 20:22:39.152269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.152171 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm" event={"ID":"432a515f-31a2-4ca1-9d3e-d1624bbdced7","Type":"ContainerDied","Data":"fb16ea62a5b8c2f8b20423bfe2054e163999c6164f6765ab535f0920a6f6f7f5"} Sep 10 20:22:39.152269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.152195 2570 scope.go:117] "RemoveContainer" containerID="c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0" Sep 10 20:22:39.162335 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.162317 2570 scope.go:117] "RemoveContainer" containerID="d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624" Sep 10 20:22:39.170400 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.170382 2570 scope.go:117] "RemoveContainer" containerID="c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0" Sep 10 20:22:39.170685 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:22:39.170664 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0\": container with ID starting with c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0 not found: ID does not exist" containerID="c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0" Sep 10 20:22:39.170746 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.170696 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0"} err="failed to get container status \"c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0\": rpc error: code = NotFound desc = could not find container \"c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0\": container with ID starting with c1ad9075e84d4c09ed3bded0cfd2d28d66159e438a964074360638201c9509b0 not found: ID does not exist" Sep 10 20:22:39.170746 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.170713 2570 scope.go:117] "RemoveContainer" containerID="d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624" Sep 10 20:22:39.170965 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:22:39.170947 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624\": container with ID starting with d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624 not found: ID does not exist" containerID="d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624" Sep 10 20:22:39.171001 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.170972 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624"} err="failed to get container status \"d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624\": rpc error: code = NotFound desc = could not find container \"d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624\": container with ID starting with d7412f2662d60824af7b8d0bb39ee9f9c64ce94e63795738a2c4a5da8921f624 not found: ID does not exist" Sep 10 20:22:39.175583 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.175562 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm"] Sep 10 20:22:39.179329 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.179307 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-316fde-openshift-default-5856b4c7bd-httqm"] Sep 10 20:22:39.767358 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:39.767325 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" path="/var/lib/kubelet/pods/432a515f-31a2-4ca1-9d3e-d1624bbdced7/volumes" Sep 10 20:22:44.348143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.348103 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c"] Sep 10 20:22:44.348599 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.348366 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" podUID="3b424483-1c6f-495f-b9a6-d882a9b30bea" containerName="payload-pre-processing" containerID="cri-o://bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154" gracePeriod=30 Sep 10 20:22:44.619714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.619647 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:22:44.639724 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.639697 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zx5n7\" (UniqueName: \"kubernetes.io/projected/3b424483-1c6f-495f-b9a6-d882a9b30bea-kube-api-access-zx5n7\") pod \"3b424483-1c6f-495f-b9a6-d882a9b30bea\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " Sep 10 20:22:44.639894 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.639837 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b424483-1c6f-495f-b9a6-d882a9b30bea-plugins-config-volume\") pod \"3b424483-1c6f-495f-b9a6-d882a9b30bea\" (UID: \"3b424483-1c6f-495f-b9a6-d882a9b30bea\") " Sep 10 20:22:44.640281 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.640252 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b424483-1c6f-495f-b9a6-d882a9b30bea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b424483-1c6f-495f-b9a6-d882a9b30bea" (UID: "3b424483-1c6f-495f-b9a6-d882a9b30bea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:22:44.641927 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.641895 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b424483-1c6f-495f-b9a6-d882a9b30bea-kube-api-access-zx5n7" (OuterVolumeSpecName: "kube-api-access-zx5n7") pod "3b424483-1c6f-495f-b9a6-d882a9b30bea" (UID: "3b424483-1c6f-495f-b9a6-d882a9b30bea"). InnerVolumeSpecName "kube-api-access-zx5n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:22:44.740727 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.740697 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b424483-1c6f-495f-b9a6-d882a9b30bea-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:44.740727 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:44.740721 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx5n7\" (UniqueName: \"kubernetes.io/projected/3b424483-1c6f-495f-b9a6-d882a9b30bea-kube-api-access-zx5n7\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:45.189737 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.189702 2570 generic.go:356] "Generic (PLEG): container finished" podID="3b424483-1c6f-495f-b9a6-d882a9b30bea" containerID="bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154" exitCode=0 Sep 10 20:22:45.189921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.189746 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" Sep 10 20:22:45.189921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.189785 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" event={"ID":"3b424483-1c6f-495f-b9a6-d882a9b30bea","Type":"ContainerDied","Data":"bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154"} Sep 10 20:22:45.189921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.189835 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c" event={"ID":"3b424483-1c6f-495f-b9a6-d882a9b30bea","Type":"ContainerDied","Data":"454e3eb96c5fe87978d4718150605d6e1fe43e3a6572472688aa3d42d5683c20"} Sep 10 20:22:45.189921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.189857 2570 scope.go:117] "RemoveContainer" containerID="bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154" Sep 10 20:22:45.199435 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.199420 2570 scope.go:117] "RemoveContainer" containerID="bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154" Sep 10 20:22:45.199708 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:22:45.199692 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154\": container with ID starting with bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154 not found: ID does not exist" containerID="bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154" Sep 10 20:22:45.199777 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.199714 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154"} err="failed to get container status \"bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154\": rpc error: code = NotFound desc = could not find container \"bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154\": container with ID starting with bd87a352b59bae1a475ee0b421113b1542dba48d3d8146a611f3132717370154 not found: ID does not exist" Sep 10 20:22:45.212614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.212590 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c"] Sep 10 20:22:45.215933 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.215911 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-82457a-558cc78c8b-vhm7c"] Sep 10 20:22:45.766177 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:45.766148 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b424483-1c6f-495f-b9a6-d882a9b30bea" path="/var/lib/kubelet/pods/3b424483-1c6f-495f-b9a6-d882a9b30bea/volumes" Sep 10 20:22:50.953669 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:50.953633 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp"] Sep 10 20:22:50.954143 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:50.953847 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" containerID="cri-o://76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14" gracePeriod=30 Sep 10 20:22:56.209513 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.209484 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:22:56.252346 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.252316 2570 generic.go:356] "Generic (PLEG): container finished" podID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerID="76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14" exitCode=0 Sep 10 20:22:56.252520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.252378 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" Sep 10 20:22:56.252520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.252407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" event={"ID":"8b9c0a83-65c4-4ccc-bffd-01e0318249f8","Type":"ContainerDied","Data":"76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14"} Sep 10 20:22:56.252520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.252477 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp" event={"ID":"8b9c0a83-65c4-4ccc-bffd-01e0318249f8","Type":"ContainerDied","Data":"91d19a54993bddf9ddf951eb971325293e8405163697deeb0f9b0777b5dd0ba5"} Sep 10 20:22:56.252520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.252506 2570 scope.go:117] "RemoveContainer" containerID="76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14" Sep 10 20:22:56.262406 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.262386 2570 scope.go:117] "RemoveContainer" containerID="b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8" Sep 10 20:22:56.270760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.270743 2570 scope.go:117] "RemoveContainer" containerID="76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14" Sep 10 20:22:56.271025 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:22:56.271009 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14\": container with ID starting with 76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14 not found: ID does not exist" containerID="76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14" Sep 10 20:22:56.271089 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.271034 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14"} err="failed to get container status \"76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14\": rpc error: code = NotFound desc = could not find container \"76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14\": container with ID starting with 76173f9629f880fc4bffe4bf574bafacd862ea7e6ea6ce440d691a5108d0ca14 not found: ID does not exist" Sep 10 20:22:56.271089 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.271050 2570 scope.go:117] "RemoveContainer" containerID="b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8" Sep 10 20:22:56.271299 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:22:56.271281 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8\": container with ID starting with b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8 not found: ID does not exist" containerID="b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8" Sep 10 20:22:56.271353 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.271308 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8"} err="failed to get container status \"b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8\": rpc error: code = NotFound desc = could not find container \"b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8\": container with ID starting with b93c02fedf577cde585d0951bf9ee3cf502cc3ba2f23f01e65f5566a196f0ed8 not found: ID does not exist" Sep 10 20:22:56.357953 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.357918 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-token\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.357971 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-credential-socket\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.357989 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-envoy\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358047 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlqm7\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-kube-api-access-wlqm7\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358091 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-certs\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358117 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-socket\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358159 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istiod-ca-cert\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358476 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358184 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-podinfo\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358476 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358224 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-data\") pod \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\" (UID: \"8b9c0a83-65c4-4ccc-bffd-01e0318249f8\") " Sep 10 20:22:56.358476 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358321 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:56.358620 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358562 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:56.358620 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358597 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:22:56.358714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358654 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.358714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358660 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:56.358714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358672 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.358714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.358688 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:56.360270 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.360243 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:22:56.360372 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.360268 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:22:56.360421 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.360394 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:22:56.360458 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.360443 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-kube-api-access-wlqm7" (OuterVolumeSpecName: "kube-api-access-wlqm7") pod "8b9c0a83-65c4-4ccc-bffd-01e0318249f8" (UID: "8b9c0a83-65c4-4ccc-bffd-01e0318249f8"). InnerVolumeSpecName "kube-api-access-wlqm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:22:56.459682 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459652 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlqm7\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-kube-api-access-wlqm7\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.459682 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459679 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.459682 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459689 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.459919 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459697 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.459919 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459706 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.459919 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459714 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.459919 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.459722 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9c0a83-65c4-4ccc-bffd-01e0318249f8-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:22:56.576301 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.576269 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp"] Sep 10 20:22:56.580776 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:56.580752 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-82457a-openshift-default-8cc4c87bb-7mjpp"] Sep 10 20:22:57.766138 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:22:57.766107 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" path="/var/lib/kubelet/pods/8b9c0a83-65c4-4ccc-bffd-01e0318249f8/volumes" Sep 10 20:23:01.911598 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:01.911563 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc"] Sep 10 20:23:01.912207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:01.911865 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" podUID="587e9320-e77b-4907-ae02-4eae39ab565e" containerName="payload-processing" containerID="cri-o://7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca" gracePeriod=30 Sep 10 20:23:01.918710 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:01.918675 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq"] Sep 10 20:23:01.920047 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:01.919977 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" podUID="d924d5ac-ac01-4879-8dfb-a4914e21e77a" containerName="payload-pre-processing" containerID="cri-o://1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5" gracePeriod=30 Sep 10 20:23:02.179171 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.179141 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:23:02.213004 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.212978 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:23:02.285508 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.285467 2570 generic.go:356] "Generic (PLEG): container finished" podID="d924d5ac-ac01-4879-8dfb-a4914e21e77a" containerID="1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5" exitCode=0 Sep 10 20:23:02.285719 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.285544 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" Sep 10 20:23:02.285719 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.285585 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" event={"ID":"d924d5ac-ac01-4879-8dfb-a4914e21e77a","Type":"ContainerDied","Data":"1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5"} Sep 10 20:23:02.285719 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.285644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq" event={"ID":"d924d5ac-ac01-4879-8dfb-a4914e21e77a","Type":"ContainerDied","Data":"d1ca4693e6228cb7ea569b2b76cc88af95d9db284bac317c1ee8726c08c70014"} Sep 10 20:23:02.285719 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.285680 2570 scope.go:117] "RemoveContainer" containerID="1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5" Sep 10 20:23:02.287133 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.287112 2570 generic.go:356] "Generic (PLEG): container finished" podID="587e9320-e77b-4907-ae02-4eae39ab565e" containerID="7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca" exitCode=0 Sep 10 20:23:02.287267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.287156 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" Sep 10 20:23:02.287267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.287168 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" event={"ID":"587e9320-e77b-4907-ae02-4eae39ab565e","Type":"ContainerDied","Data":"7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca"} Sep 10 20:23:02.287267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.287209 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc" event={"ID":"587e9320-e77b-4907-ae02-4eae39ab565e","Type":"ContainerDied","Data":"868e1bb7d2fa6e18871278d48e5a3a1bad5b9ddc9ac5ae2bdc69e024a4037116"} Sep 10 20:23:02.296019 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.295998 2570 scope.go:117] "RemoveContainer" containerID="1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5" Sep 10 20:23:02.296268 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:23:02.296249 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5\": container with ID starting with 1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5 not found: ID does not exist" containerID="1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5" Sep 10 20:23:02.296340 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.296278 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5"} err="failed to get container status \"1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5\": rpc error: code = NotFound desc = could not find container \"1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5\": container with ID starting with 1e74a0fc65b68b1f4340b9eeff97ae7352612b1d8ff398533161feb25263f8c5 not found: ID does not exist" Sep 10 20:23:02.296340 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.296294 2570 scope.go:117] "RemoveContainer" containerID="7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca" Sep 10 20:23:02.306524 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.306507 2570 scope.go:117] "RemoveContainer" containerID="7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca" Sep 10 20:23:02.307090 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:23:02.307052 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca\": container with ID starting with 7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca not found: ID does not exist" containerID="7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca" Sep 10 20:23:02.307209 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.307104 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca"} err="failed to get container status \"7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca\": rpc error: code = NotFound desc = could not find container \"7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca\": container with ID starting with 7067732185ff7cbfc9c708099f8f4e2a7435f534efecd67e7ebcd3165819afca not found: ID does not exist" Sep 10 20:23:02.317417 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.317398 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d924d5ac-ac01-4879-8dfb-a4914e21e77a-plugins-config-volume\") pod \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " Sep 10 20:23:02.317525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.317431 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lrrj\" (UniqueName: \"kubernetes.io/projected/587e9320-e77b-4907-ae02-4eae39ab565e-kube-api-access-8lrrj\") pod \"587e9320-e77b-4907-ae02-4eae39ab565e\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " Sep 10 20:23:02.317525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.317458 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv9l9\" (UniqueName: \"kubernetes.io/projected/d924d5ac-ac01-4879-8dfb-a4914e21e77a-kube-api-access-mv9l9\") pod \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\" (UID: \"d924d5ac-ac01-4879-8dfb-a4914e21e77a\") " Sep 10 20:23:02.317525 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.317480 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587e9320-e77b-4907-ae02-4eae39ab565e-plugins-config-volume\") pod \"587e9320-e77b-4907-ae02-4eae39ab565e\" (UID: \"587e9320-e77b-4907-ae02-4eae39ab565e\") " Sep 10 20:23:02.317846 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.317820 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d924d5ac-ac01-4879-8dfb-a4914e21e77a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d924d5ac-ac01-4879-8dfb-a4914e21e77a" (UID: "d924d5ac-ac01-4879-8dfb-a4914e21e77a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:23:02.317927 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.317850 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/587e9320-e77b-4907-ae02-4eae39ab565e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "587e9320-e77b-4907-ae02-4eae39ab565e" (UID: "587e9320-e77b-4907-ae02-4eae39ab565e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:23:02.319413 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.319392 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587e9320-e77b-4907-ae02-4eae39ab565e-kube-api-access-8lrrj" (OuterVolumeSpecName: "kube-api-access-8lrrj") pod "587e9320-e77b-4907-ae02-4eae39ab565e" (UID: "587e9320-e77b-4907-ae02-4eae39ab565e"). InnerVolumeSpecName "kube-api-access-8lrrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:23:02.319485 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.319468 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d924d5ac-ac01-4879-8dfb-a4914e21e77a-kube-api-access-mv9l9" (OuterVolumeSpecName: "kube-api-access-mv9l9") pod "d924d5ac-ac01-4879-8dfb-a4914e21e77a" (UID: "d924d5ac-ac01-4879-8dfb-a4914e21e77a"). InnerVolumeSpecName "kube-api-access-mv9l9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:23:02.419040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.418955 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d924d5ac-ac01-4879-8dfb-a4914e21e77a-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:02.419040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.418985 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lrrj\" (UniqueName: \"kubernetes.io/projected/587e9320-e77b-4907-ae02-4eae39ab565e-kube-api-access-8lrrj\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:02.419040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.418994 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv9l9\" (UniqueName: \"kubernetes.io/projected/d924d5ac-ac01-4879-8dfb-a4914e21e77a-kube-api-access-mv9l9\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:02.419040 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.419003 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587e9320-e77b-4907-ae02-4eae39ab565e-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:02.608826 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.608792 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq"] Sep 10 20:23:02.612303 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.612275 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-591d7c-6649cddbc8-zxbbq"] Sep 10 20:23:02.622719 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.622691 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc"] Sep 10 20:23:02.626147 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:02.626125 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-591d7c-7cfdc9d449-fjgkc"] Sep 10 20:23:03.766039 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:03.766007 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="587e9320-e77b-4907-ae02-4eae39ab565e" path="/var/lib/kubelet/pods/587e9320-e77b-4907-ae02-4eae39ab565e/volumes" Sep 10 20:23:03.766447 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:03.766388 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d924d5ac-ac01-4879-8dfb-a4914e21e77a" path="/var/lib/kubelet/pods/d924d5ac-ac01-4879-8dfb-a4914e21e77a/volumes" Sep 10 20:23:09.097108 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097058 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q"] Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097455 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="587e9320-e77b-4907-ae02-4eae39ab565e" containerName="payload-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097466 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="587e9320-e77b-4907-ae02-4eae39ab565e" containerName="payload-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097480 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097487 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097506 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" containerName="payload-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097514 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" containerName="payload-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097522 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097531 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097553 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d924d5ac-ac01-4879-8dfb-a4914e21e77a" containerName="payload-pre-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097561 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d924d5ac-ac01-4879-8dfb-a4914e21e77a" containerName="payload-pre-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097570 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097578 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097593 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b424483-1c6f-495f-b9a6-d882a9b30bea" containerName="payload-pre-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097601 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b424483-1c6f-495f-b9a6-d882a9b30bea" containerName="payload-pre-processing" Sep 10 20:23:09.097613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097617 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097626 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097728 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097737 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b424483-1c6f-495f-b9a6-d882a9b30bea" containerName="payload-pre-processing" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097744 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097750 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097759 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097765 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d924d5ac-ac01-4879-8dfb-a4914e21e77a" containerName="payload-pre-processing" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097773 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6b82e7-805a-49a8-a0d3-d5f37a961fa1" containerName="payload-processing" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097780 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="432a515f-31a2-4ca1-9d3e-d1624bbdced7" containerName="istio-proxy" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097787 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="587e9320-e77b-4907-ae02-4eae39ab565e" containerName="payload-processing" Sep 10 20:23:09.098437 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.097793 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" Sep 10 20:23:09.100726 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.100703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:09.103266 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.103243 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:23:09.103366 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.103243 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zdpml\"" Sep 10 20:23:09.109736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.109712 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q"] Sep 10 20:23:09.287543 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.287502 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72daee81-26c8-446b-b5cd-147f3c0c2958-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q\" (UID: \"72daee81-26c8-446b-b5cd-147f3c0c2958\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:09.388463 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.388380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72daee81-26c8-446b-b5cd-147f3c0c2958-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q\" (UID: \"72daee81-26c8-446b-b5cd-147f3c0c2958\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:09.388976 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.388956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72daee81-26c8-446b-b5cd-147f3c0c2958-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q\" (UID: \"72daee81-26c8-446b-b5cd-147f3c0c2958\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:09.413246 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.413225 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:09.681956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:09.681931 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q"] Sep 10 20:23:09.683958 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:23:09.683931 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72daee81_26c8_446b_b5cd_147f3c0c2958.slice/crio-5e2a5637ff35f2d4dae5fe2b7556da9cfe656a6a7c97174b0a909a61dffe60cf WatchSource:0}: Error finding container 5e2a5637ff35f2d4dae5fe2b7556da9cfe656a6a7c97174b0a909a61dffe60cf: Status 404 returned error can't find the container with id 5e2a5637ff35f2d4dae5fe2b7556da9cfe656a6a7c97174b0a909a61dffe60cf Sep 10 20:23:10.334167 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:10.334128 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" event={"ID":"72daee81-26c8-446b-b5cd-147f3c0c2958","Type":"ContainerStarted","Data":"9a799c3ffecfdcdf88075dff4ed4627ea79812e922bff931df34b9323e1d5354"} Sep 10 20:23:10.334598 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:10.334183 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" event={"ID":"72daee81-26c8-446b-b5cd-147f3c0c2958","Type":"ContainerStarted","Data":"5e2a5637ff35f2d4dae5fe2b7556da9cfe656a6a7c97174b0a909a61dffe60cf"} Sep 10 20:23:10.355820 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:10.355759 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" podStartSLOduration=1.355743478 podStartE2EDuration="1.355743478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:23:10.354353 +0000 UTC m=+1759.075300163" watchObservedRunningTime="2026-09-10 20:23:10.355743478 +0000 UTC m=+1759.076690598" Sep 10 20:23:11.341787 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:11.341749 2570 generic.go:356] "Generic (PLEG): container finished" podID="72daee81-26c8-446b-b5cd-147f3c0c2958" containerID="9a799c3ffecfdcdf88075dff4ed4627ea79812e922bff931df34b9323e1d5354" exitCode=0 Sep 10 20:23:11.342226 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:11.341785 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" event={"ID":"72daee81-26c8-446b-b5cd-147f3c0c2958","Type":"ContainerDied","Data":"9a799c3ffecfdcdf88075dff4ed4627ea79812e922bff931df34b9323e1d5354"} Sep 10 20:23:12.478204 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:12.478177 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:12.518255 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:12.518225 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72daee81-26c8-446b-b5cd-147f3c0c2958-maas-api-service-ca\") pod \"72daee81-26c8-446b-b5cd-147f3c0c2958\" (UID: \"72daee81-26c8-446b-b5cd-147f3c0c2958\") " Sep 10 20:23:12.518613 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:12.518590 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72daee81-26c8-446b-b5cd-147f3c0c2958-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72daee81-26c8-446b-b5cd-147f3c0c2958" (UID: "72daee81-26c8-446b-b5cd-147f3c0c2958"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:23:12.619282 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:12.619218 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72daee81-26c8-446b-b5cd-147f3c0c2958-maas-api-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:13.353123 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:13.353098 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" Sep 10 20:23:13.353312 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:13.353087 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q" event={"ID":"72daee81-26c8-446b-b5cd-147f3c0c2958","Type":"ContainerDied","Data":"5e2a5637ff35f2d4dae5fe2b7556da9cfe656a6a7c97174b0a909a61dffe60cf"} Sep 10 20:23:13.353312 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:13.353221 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e2a5637ff35f2d4dae5fe2b7556da9cfe656a6a7c97174b0a909a61dffe60cf" Sep 10 20:23:19.606491 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.606455 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98"] Sep 10 20:23:19.606955 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.606675 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" podUID="6ef9b018-dae0-497b-9655-b940c5d142fd" containerName="payload-pre-processing" containerID="cri-o://0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b" gracePeriod=30 Sep 10 20:23:19.867404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.867333 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:23:19.874095 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.874013 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q"] Sep 10 20:23:19.876002 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.875977 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6550b4-bzs4q"] Sep 10 20:23:19.886967 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.886888 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkpwn\" (UniqueName: \"kubernetes.io/projected/6ef9b018-dae0-497b-9655-b940c5d142fd-kube-api-access-rkpwn\") pod \"6ef9b018-dae0-497b-9655-b940c5d142fd\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " Sep 10 20:23:19.886967 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.886947 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ef9b018-dae0-497b-9655-b940c5d142fd-plugins-config-volume\") pod \"6ef9b018-dae0-497b-9655-b940c5d142fd\" (UID: \"6ef9b018-dae0-497b-9655-b940c5d142fd\") " Sep 10 20:23:19.887662 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.887636 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ef9b018-dae0-497b-9655-b940c5d142fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ef9b018-dae0-497b-9655-b940c5d142fd" (UID: "6ef9b018-dae0-497b-9655-b940c5d142fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:23:19.889701 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.889676 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef9b018-dae0-497b-9655-b940c5d142fd-kube-api-access-rkpwn" (OuterVolumeSpecName: "kube-api-access-rkpwn") pod "6ef9b018-dae0-497b-9655-b940c5d142fd" (UID: "6ef9b018-dae0-497b-9655-b940c5d142fd"). InnerVolumeSpecName "kube-api-access-rkpwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:23:19.988746 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.988704 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkpwn\" (UniqueName: \"kubernetes.io/projected/6ef9b018-dae0-497b-9655-b940c5d142fd-kube-api-access-rkpwn\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:19.988746 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:19.988743 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ef9b018-dae0-497b-9655-b940c5d142fd-plugins-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:20.391910 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.391876 2570 generic.go:356] "Generic (PLEG): container finished" podID="6ef9b018-dae0-497b-9655-b940c5d142fd" containerID="0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b" exitCode=0 Sep 10 20:23:20.392099 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.391923 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" Sep 10 20:23:20.392099 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.391963 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" event={"ID":"6ef9b018-dae0-497b-9655-b940c5d142fd","Type":"ContainerDied","Data":"0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b"} Sep 10 20:23:20.392099 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.392017 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98" event={"ID":"6ef9b018-dae0-497b-9655-b940c5d142fd","Type":"ContainerDied","Data":"2b83cfce81f3539011ea389a50116115e44def2f22876cfab3fa3631b01108cc"} Sep 10 20:23:20.392099 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.392040 2570 scope.go:117] "RemoveContainer" containerID="0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b" Sep 10 20:23:20.401182 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.401164 2570 scope.go:117] "RemoveContainer" containerID="0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b" Sep 10 20:23:20.401467 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:23:20.401449 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b\": container with ID starting with 0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b not found: ID does not exist" containerID="0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b" Sep 10 20:23:20.401523 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.401476 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b"} err="failed to get container status \"0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b\": rpc error: code = NotFound desc = could not find container \"0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b\": container with ID starting with 0089ad2d305d322adc8c60ade3890e85bbd9f69c1c8be4eb1766f2c1843b800b not found: ID does not exist" Sep 10 20:23:20.413430 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.413410 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98"] Sep 10 20:23:20.417095 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:20.417060 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6550b4-d97b78bdb-75w98"] Sep 10 20:23:21.766866 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:21.766833 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ef9b018-dae0-497b-9655-b940c5d142fd" path="/var/lib/kubelet/pods/6ef9b018-dae0-497b-9655-b940c5d142fd/volumes" Sep 10 20:23:21.767269 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:21.767190 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72daee81-26c8-446b-b5cd-147f3c0c2958" path="/var/lib/kubelet/pods/72daee81-26c8-446b-b5cd-147f3c0c2958/volumes" Sep 10 20:23:26.542543 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:26.542506 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v"] Sep 10 20:23:26.543036 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:26.542759 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" containerID="cri-o://3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740" gracePeriod=30 Sep 10 20:23:26.870422 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:26.870329 2570 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 20:23:26.870422 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:26.870390 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 20:23:29.351541 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.351502 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc475458-vq44r"] Sep 10 20:23:29.351956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.351748 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fc475458-vq44r" podUID="e8532c6d-0e25-46da-b999-cbffa8cda1bb" containerName="manager" containerID="cri-o://4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa" gracePeriod=30 Sep 10 20:23:29.607992 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.607964 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:23:29.681945 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.681913 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-metrics-certs\") pod \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " Sep 10 20:23:29.682129 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.681986 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert\") pod \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " Sep 10 20:23:29.682185 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.682133 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg5b6\" (UniqueName: \"kubernetes.io/projected/e8532c6d-0e25-46da-b999-cbffa8cda1bb-kube-api-access-kg5b6\") pod \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\" (UID: \"e8532c6d-0e25-46da-b999-cbffa8cda1bb\") " Sep 10 20:23:29.684158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.684122 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8532c6d-0e25-46da-b999-cbffa8cda1bb" (UID: "e8532c6d-0e25-46da-b999-cbffa8cda1bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:23:29.684274 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.684185 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8532c6d-0e25-46da-b999-cbffa8cda1bb-kube-api-access-kg5b6" (OuterVolumeSpecName: "kube-api-access-kg5b6") pod "e8532c6d-0e25-46da-b999-cbffa8cda1bb" (UID: "e8532c6d-0e25-46da-b999-cbffa8cda1bb"). InnerVolumeSpecName "kube-api-access-kg5b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:23:29.684409 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.684375 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e8532c6d-0e25-46da-b999-cbffa8cda1bb" (UID: "e8532c6d-0e25-46da-b999-cbffa8cda1bb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:23:29.783473 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.783442 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg5b6\" (UniqueName: \"kubernetes.io/projected/e8532c6d-0e25-46da-b999-cbffa8cda1bb-kube-api-access-kg5b6\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:29.783473 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.783470 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:29.783473 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:29.783479 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8532c6d-0e25-46da-b999-cbffa8cda1bb-webhook-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:30.447666 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.447629 2570 generic.go:356] "Generic (PLEG): container finished" podID="e8532c6d-0e25-46da-b999-cbffa8cda1bb" containerID="4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa" exitCode=0 Sep 10 20:23:30.448180 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.447675 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-vq44r" Sep 10 20:23:30.448180 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.447691 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-vq44r" event={"ID":"e8532c6d-0e25-46da-b999-cbffa8cda1bb","Type":"ContainerDied","Data":"4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa"} Sep 10 20:23:30.448180 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.447743 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-vq44r" event={"ID":"e8532c6d-0e25-46da-b999-cbffa8cda1bb","Type":"ContainerDied","Data":"3aa12dc646e9c42f1527051e5827d4129f1f175e13e79018d81c7172a3df2dac"} Sep 10 20:23:30.448180 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.447768 2570 scope.go:117] "RemoveContainer" containerID="4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa" Sep 10 20:23:30.457616 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.457587 2570 scope.go:117] "RemoveContainer" containerID="4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa" Sep 10 20:23:30.457870 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:23:30.457850 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa\": container with ID starting with 4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa not found: ID does not exist" containerID="4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa" Sep 10 20:23:30.457915 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.457878 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa"} err="failed to get container status \"4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa\": rpc error: code = NotFound desc = could not find container \"4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa\": container with ID starting with 4d87468e983343efe7da58fe772569cbb754922f099015629262cef806e1e8aa not found: ID does not exist" Sep 10 20:23:30.464814 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.464790 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc475458-vq44r"] Sep 10 20:23:30.466666 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:30.466644 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fc475458-vq44r"] Sep 10 20:23:31.766459 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.766430 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8532c6d-0e25-46da-b999-cbffa8cda1bb" path="/var/lib/kubelet/pods/e8532c6d-0e25-46da-b999-cbffa8cda1bb/volumes" Sep 10 20:23:31.797538 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.797516 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:23:31.903629 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903540 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz22c\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-kube-api-access-lz22c\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903629 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903592 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-socket\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903634 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-istio-token\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903693 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-data\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903728 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-credential-socket\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903761 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e618051e-29b7-4ede-be2a-659f5c484acd-istio-podinfo\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903808 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-certs\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.903872 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903851 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-envoy\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.904224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903905 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e618051e-29b7-4ede-be2a-659f5c484acd-istiod-ca-cert\") pod \"e618051e-29b7-4ede-be2a-659f5c484acd\" (UID: \"e618051e-29b7-4ede-be2a-659f5c484acd\") " Sep 10 20:23:31.904224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.903946 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:23:31.904224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904023 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-data" (OuterVolumeSpecName: "istio-data") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:23:31.904224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904037 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:23:31.904224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904205 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:23:31.904469 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904338 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:31.904469 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904355 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-data\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:31.904469 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904369 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-credential-socket\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:31.904469 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904372 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e618051e-29b7-4ede-be2a-659f5c484acd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:23:31.904469 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.904384 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-workload-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:31.906184 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.906162 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-istio-token" (OuterVolumeSpecName: "istio-token") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:23:31.906318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.906210 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-kube-api-access-lz22c" (OuterVolumeSpecName: "kube-api-access-lz22c") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "kube-api-access-lz22c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:23:31.906318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.906207 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:23:31.906318 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:31.906264 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e618051e-29b7-4ede-be2a-659f5c484acd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e618051e-29b7-4ede-be2a-659f5c484acd" (UID: "e618051e-29b7-4ede-be2a-659f5c484acd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 20:23:32.005636 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.005594 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-istio-token\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:32.005636 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.005627 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e618051e-29b7-4ede-be2a-659f5c484acd-istio-podinfo\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:32.005636 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.005640 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e618051e-29b7-4ede-be2a-659f5c484acd-istio-envoy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:32.005897 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.005650 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e618051e-29b7-4ede-be2a-659f5c484acd-istiod-ca-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:32.005897 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.005662 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz22c\" (UniqueName: \"kubernetes.io/projected/e618051e-29b7-4ede-be2a-659f5c484acd-kube-api-access-lz22c\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:23:32.461464 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.461429 2570 generic.go:356] "Generic (PLEG): container finished" podID="e618051e-29b7-4ede-be2a-659f5c484acd" containerID="3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740" exitCode=0 Sep 10 20:23:32.461647 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.461472 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" Sep 10 20:23:32.461647 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.461489 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" event={"ID":"e618051e-29b7-4ede-be2a-659f5c484acd","Type":"ContainerDied","Data":"3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740"} Sep 10 20:23:32.461647 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.461531 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v" event={"ID":"e618051e-29b7-4ede-be2a-659f5c484acd","Type":"ContainerDied","Data":"3d1e089bc39ad1f8624cda4343d7fc6e4c96e03c33d478723859339e2eb0ddce"} Sep 10 20:23:32.461647 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.461552 2570 scope.go:117] "RemoveContainer" containerID="3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740" Sep 10 20:23:32.470769 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.470749 2570 scope.go:117] "RemoveContainer" containerID="abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d" Sep 10 20:23:32.478615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.478598 2570 scope.go:117] "RemoveContainer" containerID="3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740" Sep 10 20:23:32.478889 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:23:32.478866 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740\": container with ID starting with 3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740 not found: ID does not exist" containerID="3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740" Sep 10 20:23:32.478985 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.478896 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740"} err="failed to get container status \"3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740\": rpc error: code = NotFound desc = could not find container \"3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740\": container with ID starting with 3c634050191af56a70742101c686ebce59ec77690070f1a5a57a5a7aa7ee5740 not found: ID does not exist" Sep 10 20:23:32.478985 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.478914 2570 scope.go:117] "RemoveContainer" containerID="abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d" Sep 10 20:23:32.479184 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:23:32.479169 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d\": container with ID starting with abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d not found: ID does not exist" containerID="abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d" Sep 10 20:23:32.479230 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.479188 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d"} err="failed to get container status \"abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d\": rpc error: code = NotFound desc = could not find container \"abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d\": container with ID starting with abb42d770868ba972e5c15de4ce3d97d649deb8b24ccfef5288446f1ccdf799d not found: ID does not exist" Sep 10 20:23:32.483519 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.483498 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v"] Sep 10 20:23:32.487291 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:32.487270 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6550b4-openshift-default-6fc78ddd46-8nb5v"] Sep 10 20:23:33.766063 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:33.766031 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" path="/var/lib/kubelet/pods/e618051e-29b7-4ede-be2a-659f5c484acd/volumes" Sep 10 20:23:52.105716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:23:52.105681 2570 scope.go:117] "RemoveContainer" containerID="0fd61219827b30009870c653579f40b6a9ba2a1e1ccff06898bbc93ba8a253a7" Sep 10 20:24:31.287199 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287162 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fc475458-dht6b"] Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287601 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef9b018-dae0-497b-9655-b940c5d142fd" containerName="payload-pre-processing" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287614 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef9b018-dae0-497b-9655-b940c5d142fd" containerName="payload-pre-processing" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287623 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287628 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4a2eb2-dee4-49f7-8989-bb3071d310c2" containerName="istio-proxy" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287636 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287641 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287658 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8532c6d-0e25-46da-b999-cbffa8cda1bb" containerName="manager" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287663 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8532c6d-0e25-46da-b999-cbffa8cda1bb" containerName="manager" Sep 10 20:24:31.287674 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287677 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72daee81-26c8-446b-b5cd-147f3c0c2958" containerName="revoke-keys" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287683 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="72daee81-26c8-446b-b5cd-147f3c0c2958" containerName="revoke-keys" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287690 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287695 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9c0a83-65c4-4ccc-bffd-01e0318249f8" containerName="istio-proxy" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287748 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8532c6d-0e25-46da-b999-cbffa8cda1bb" containerName="manager" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287757 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="72daee81-26c8-446b-b5cd-147f3c0c2958" containerName="revoke-keys" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287764 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287771 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef9b018-dae0-497b-9655-b940c5d142fd" containerName="payload-pre-processing" Sep 10 20:24:31.287968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.287777 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" Sep 10 20:24:31.290829 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.290813 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.294483 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.294451 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 20:24:31.294483 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.294475 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 20:24:31.294716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.294509 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 20:24:31.294716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.294516 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9564f\"" Sep 10 20:24:31.299875 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.299842 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc475458-dht6b"] Sep 10 20:24:31.357384 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.357349 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-webhook-cert\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.357561 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.357423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9rbb\" (UniqueName: \"kubernetes.io/projected/778d39e6-5dbb-46d3-9fce-de60bebb9a26-kube-api-access-z9rbb\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.357561 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.357471 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-metrics-certs\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.458287 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.458253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9rbb\" (UniqueName: \"kubernetes.io/projected/778d39e6-5dbb-46d3-9fce-de60bebb9a26-kube-api-access-z9rbb\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.458473 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.458336 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-metrics-certs\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.458473 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.458371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-webhook-cert\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.460878 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.460847 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-webhook-cert\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.460989 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.460879 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-metrics-certs\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.467324 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.467302 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9rbb\" (UniqueName: \"kubernetes.io/projected/778d39e6-5dbb-46d3-9fce-de60bebb9a26-kube-api-access-z9rbb\") pod \"maas-controller-7fc475458-dht6b\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.602117 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.602011 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:31.877513 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.877482 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc475458-dht6b"] Sep 10 20:24:31.879827 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:24:31.879801 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778d39e6_5dbb_46d3_9fce_de60bebb9a26.slice/crio-3ba2b2f8f1bb08a86e9f29967d4e6bc4f8f6d809abb5034f0d87cf23e4cfc056 WatchSource:0}: Error finding container 3ba2b2f8f1bb08a86e9f29967d4e6bc4f8f6d809abb5034f0d87cf23e4cfc056: Status 404 returned error can't find the container with id 3ba2b2f8f1bb08a86e9f29967d4e6bc4f8f6d809abb5034f0d87cf23e4cfc056 Sep 10 20:24:31.881119 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:31.881097 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 20:24:32.778746 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:32.778712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-dht6b" event={"ID":"778d39e6-5dbb-46d3-9fce-de60bebb9a26","Type":"ContainerStarted","Data":"f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52"} Sep 10 20:24:32.779158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:32.778762 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-dht6b" event={"ID":"778d39e6-5dbb-46d3-9fce-de60bebb9a26","Type":"ContainerStarted","Data":"3ba2b2f8f1bb08a86e9f29967d4e6bc4f8f6d809abb5034f0d87cf23e4cfc056"} Sep 10 20:24:32.779158 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:32.778857 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:32.799454 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:32.799404 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fc475458-dht6b" podStartSLOduration=1.321465465 podStartE2EDuration="1.799390074s" totalImagesPullingTime="477.924609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:24:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:24:32.798259078 +0000 UTC m=+1841.519206202" watchObservedRunningTime="2026-09-10 20:24:32.799390074 +0000 UTC m=+1841.520337193" Sep 10 20:24:43.785531 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:43.785499 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:51.093458 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.093414 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc475458-dht6b"] Sep 10 20:24:51.093888 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.093673 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fc475458-dht6b" podUID="778d39e6-5dbb-46d3-9fce-de60bebb9a26" containerName="manager" containerID="cri-o://f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52" gracePeriod=30 Sep 10 20:24:51.333363 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.333337 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:51.439015 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.438929 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-webhook-cert\") pod \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " Sep 10 20:24:51.439200 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.439021 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-metrics-certs\") pod \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " Sep 10 20:24:51.439200 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.439049 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9rbb\" (UniqueName: \"kubernetes.io/projected/778d39e6-5dbb-46d3-9fce-de60bebb9a26-kube-api-access-z9rbb\") pod \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\" (UID: \"778d39e6-5dbb-46d3-9fce-de60bebb9a26\") " Sep 10 20:24:51.441124 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.441091 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778d39e6-5dbb-46d3-9fce-de60bebb9a26-kube-api-access-z9rbb" (OuterVolumeSpecName: "kube-api-access-z9rbb") pod "778d39e6-5dbb-46d3-9fce-de60bebb9a26" (UID: "778d39e6-5dbb-46d3-9fce-de60bebb9a26"). InnerVolumeSpecName "kube-api-access-z9rbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:24:51.441242 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.441145 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "778d39e6-5dbb-46d3-9fce-de60bebb9a26" (UID: "778d39e6-5dbb-46d3-9fce-de60bebb9a26"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:24:51.441242 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.441177 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "778d39e6-5dbb-46d3-9fce-de60bebb9a26" (UID: "778d39e6-5dbb-46d3-9fce-de60bebb9a26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:24:51.539862 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.539829 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:24:51.539862 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.539858 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9rbb\" (UniqueName: \"kubernetes.io/projected/778d39e6-5dbb-46d3-9fce-de60bebb9a26-kube-api-access-z9rbb\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:24:51.539862 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.539868 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/778d39e6-5dbb-46d3-9fce-de60bebb9a26-webhook-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:24:51.874579 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.874548 2570 generic.go:356] "Generic (PLEG): container finished" podID="778d39e6-5dbb-46d3-9fce-de60bebb9a26" containerID="f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52" exitCode=0 Sep 10 20:24:51.874744 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.874594 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-dht6b" Sep 10 20:24:51.874744 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.874631 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-dht6b" event={"ID":"778d39e6-5dbb-46d3-9fce-de60bebb9a26","Type":"ContainerDied","Data":"f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52"} Sep 10 20:24:51.874744 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.874681 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-dht6b" event={"ID":"778d39e6-5dbb-46d3-9fce-de60bebb9a26","Type":"ContainerDied","Data":"3ba2b2f8f1bb08a86e9f29967d4e6bc4f8f6d809abb5034f0d87cf23e4cfc056"} Sep 10 20:24:51.874744 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.874703 2570 scope.go:117] "RemoveContainer" containerID="f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52" Sep 10 20:24:51.883614 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.883598 2570 scope.go:117] "RemoveContainer" containerID="f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52" Sep 10 20:24:51.883884 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:24:51.883865 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52\": container with ID starting with f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52 not found: ID does not exist" containerID="f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52" Sep 10 20:24:51.883967 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.883894 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52"} err="failed to get container status \"f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52\": rpc error: code = NotFound desc = could not find container \"f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52\": container with ID starting with f7b1841386140a08f94169543c5abcacd61893508650842e84dbf578658f0b52 not found: ID does not exist" Sep 10 20:24:51.893669 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.893648 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc475458-dht6b"] Sep 10 20:24:51.897307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:51.897288 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fc475458-dht6b"] Sep 10 20:24:52.154957 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.154882 2570 scope.go:117] "RemoveContainer" containerID="6f7ad47193fdf234680ac4ad13f768c979db3f3f537be8ee7fabb559ca862ba7" Sep 10 20:24:52.163237 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.163217 2570 scope.go:117] "RemoveContainer" containerID="31d34ee49b9756786d4b60800041852c23bbe0395350ec893e15a1298483f4a2" Sep 10 20:24:52.170919 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.170899 2570 scope.go:117] "RemoveContainer" containerID="4e03915b46fb2c4137e764bdf0fa59050c25f18bb1b5331932ef75c5d3fc1524" Sep 10 20:24:52.963135 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.963104 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fc475458-56m2z"] Sep 10 20:24:52.963514 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.963501 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" Sep 10 20:24:52.963558 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.963515 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e618051e-29b7-4ede-be2a-659f5c484acd" containerName="istio-proxy" Sep 10 20:24:52.963558 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.963525 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778d39e6-5dbb-46d3-9fce-de60bebb9a26" containerName="manager" Sep 10 20:24:52.963558 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.963530 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="778d39e6-5dbb-46d3-9fce-de60bebb9a26" containerName="manager" Sep 10 20:24:52.963654 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.963597 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="778d39e6-5dbb-46d3-9fce-de60bebb9a26" containerName="manager" Sep 10 20:24:52.968465 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.968444 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:52.970757 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.970731 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 20:24:52.972021 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.971999 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9564f\"" Sep 10 20:24:52.972131 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.972005 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 20:24:52.972131 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.972004 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 20:24:52.975512 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:52.975489 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc475458-56m2z"] Sep 10 20:24:53.055812 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.055779 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/336e0226-37c2-4843-a45a-64e6550a7e86-webhook-cert\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.055979 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.055845 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336e0226-37c2-4843-a45a-64e6550a7e86-metrics-certs\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.055979 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.055885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46zcx\" (UniqueName: \"kubernetes.io/projected/336e0226-37c2-4843-a45a-64e6550a7e86-kube-api-access-46zcx\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.157197 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.157161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/336e0226-37c2-4843-a45a-64e6550a7e86-webhook-cert\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.157676 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.157250 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336e0226-37c2-4843-a45a-64e6550a7e86-metrics-certs\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.157676 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.157279 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46zcx\" (UniqueName: \"kubernetes.io/projected/336e0226-37c2-4843-a45a-64e6550a7e86-kube-api-access-46zcx\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.159760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.159737 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/336e0226-37c2-4843-a45a-64e6550a7e86-webhook-cert\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.159760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.159756 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336e0226-37c2-4843-a45a-64e6550a7e86-metrics-certs\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.165521 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.165496 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46zcx\" (UniqueName: \"kubernetes.io/projected/336e0226-37c2-4843-a45a-64e6550a7e86-kube-api-access-46zcx\") pod \"maas-controller-7fc475458-56m2z\" (UID: \"336e0226-37c2-4843-a45a-64e6550a7e86\") " pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.279949 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.279913 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:53.550782 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.550754 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc475458-56m2z"] Sep 10 20:24:53.553590 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:24:53.553561 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336e0226_37c2_4843_a45a_64e6550a7e86.slice/crio-3e73fe659b47795bcd21205bf3cee0eb546d7d5607b45e7eb2d99216c63dc54c WatchSource:0}: Error finding container 3e73fe659b47795bcd21205bf3cee0eb546d7d5607b45e7eb2d99216c63dc54c: Status 404 returned error can't find the container with id 3e73fe659b47795bcd21205bf3cee0eb546d7d5607b45e7eb2d99216c63dc54c Sep 10 20:24:53.766841 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.766808 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="778d39e6-5dbb-46d3-9fce-de60bebb9a26" path="/var/lib/kubelet/pods/778d39e6-5dbb-46d3-9fce-de60bebb9a26/volumes" Sep 10 20:24:53.885969 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:53.885872 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-56m2z" event={"ID":"336e0226-37c2-4843-a45a-64e6550a7e86","Type":"ContainerStarted","Data":"3e73fe659b47795bcd21205bf3cee0eb546d7d5607b45e7eb2d99216c63dc54c"} Sep 10 20:24:54.893138 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:54.893099 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc475458-56m2z" event={"ID":"336e0226-37c2-4843-a45a-64e6550a7e86","Type":"ContainerStarted","Data":"dbf55ca1d49be5573c5531bf6b5d0b5c0e22369edd6495d8fbbdd716e62b395b"} Sep 10 20:24:54.893522 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:54.893167 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:24:54.912875 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:24:54.912826 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fc475458-56m2z" podStartSLOduration=2.438400938 podStartE2EDuration="2.912802759s" totalImagesPullingTime="474.401821ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:24:54.911371829 +0000 UTC m=+1863.632318972" watchObservedRunningTime="2026-09-10 20:24:54.912802759 +0000 UTC m=+1863.633749908" Sep 10 20:25:05.900716 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:25:05.900685 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fc475458-56m2z" Sep 10 20:26:18.978751 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:18.978714 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz"] Sep 10 20:26:18.981622 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:18.978906 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" podUID="a090ded7-60bb-484a-aedf-36b06ae6829f" containerName="manager" containerID="cri-o://92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316" gracePeriod=10 Sep 10 20:26:19.229261 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.229205 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:26:19.310793 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.310758 2570 generic.go:356] "Generic (PLEG): container finished" podID="a090ded7-60bb-484a-aedf-36b06ae6829f" containerID="92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316" exitCode=0 Sep 10 20:26:19.310956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.310806 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" Sep 10 20:26:19.310956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.310817 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" event={"ID":"a090ded7-60bb-484a-aedf-36b06ae6829f","Type":"ContainerDied","Data":"92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316"} Sep 10 20:26:19.310956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.310872 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz" event={"ID":"a090ded7-60bb-484a-aedf-36b06ae6829f","Type":"ContainerDied","Data":"36cde840aee305b0a709c493c35fa2a04e5ec148510ee617f2e222e0db6a1a2e"} Sep 10 20:26:19.310956 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.310899 2570 scope.go:117] "RemoveContainer" containerID="92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316" Sep 10 20:26:19.320312 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.320242 2570 scope.go:117] "RemoveContainer" containerID="92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316" Sep 10 20:26:19.320523 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:26:19.320504 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316\": container with ID starting with 92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316 not found: ID does not exist" containerID="92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316" Sep 10 20:26:19.320562 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.320532 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316"} err="failed to get container status \"92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316\": rpc error: code = NotFound desc = could not find container \"92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316\": container with ID starting with 92179b1c184ae6c9c991b509edb285a8041dfcaa3ca6ab81b0eebd1ce8143316 not found: ID does not exist" Sep 10 20:26:19.380640 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.380609 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a090ded7-60bb-484a-aedf-36b06ae6829f-extensions-socket-volume\") pod \"a090ded7-60bb-484a-aedf-36b06ae6829f\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " Sep 10 20:26:19.380787 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.380695 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n8s6\" (UniqueName: \"kubernetes.io/projected/a090ded7-60bb-484a-aedf-36b06ae6829f-kube-api-access-6n8s6\") pod \"a090ded7-60bb-484a-aedf-36b06ae6829f\" (UID: \"a090ded7-60bb-484a-aedf-36b06ae6829f\") " Sep 10 20:26:19.381003 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.380983 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a090ded7-60bb-484a-aedf-36b06ae6829f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a090ded7-60bb-484a-aedf-36b06ae6829f" (UID: "a090ded7-60bb-484a-aedf-36b06ae6829f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 20:26:19.382936 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.382916 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a090ded7-60bb-484a-aedf-36b06ae6829f-kube-api-access-6n8s6" (OuterVolumeSpecName: "kube-api-access-6n8s6") pod "a090ded7-60bb-484a-aedf-36b06ae6829f" (UID: "a090ded7-60bb-484a-aedf-36b06ae6829f"). InnerVolumeSpecName "kube-api-access-6n8s6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:26:19.481988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.481911 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n8s6\" (UniqueName: \"kubernetes.io/projected/a090ded7-60bb-484a-aedf-36b06ae6829f-kube-api-access-6n8s6\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:26:19.481988 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.481940 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a090ded7-60bb-484a-aedf-36b06ae6829f-extensions-socket-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:26:19.634114 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.634086 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz"] Sep 10 20:26:19.640306 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.640279 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-94hrz"] Sep 10 20:26:19.770041 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:26:19.770004 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a090ded7-60bb-484a-aedf-36b06ae6829f" path="/var/lib/kubelet/pods/a090ded7-60bb-484a-aedf-36b06ae6829f/volumes" Sep 10 20:27:19.704432 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.701451 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv"] Sep 10 20:27:19.704432 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.702408 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a090ded7-60bb-484a-aedf-36b06ae6829f" containerName="manager" Sep 10 20:27:19.704432 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.702429 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a090ded7-60bb-484a-aedf-36b06ae6829f" containerName="manager" Sep 10 20:27:19.704432 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.702603 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a090ded7-60bb-484a-aedf-36b06ae6829f" containerName="manager" Sep 10 20:27:19.706523 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.706498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:19.709414 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.709388 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 20:27:19.709414 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.709407 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 20:27:19.709615 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.709449 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fnjn8\"" Sep 10 20:27:19.717016 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.716996 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv"] Sep 10 20:27:19.733954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.733928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b50a2854-d80e-4a05-8215-e5f1a38b319a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8xjnv\" (UID: \"b50a2854-d80e-4a05-8215-e5f1a38b319a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:19.734100 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.734008 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwzbc\" (UniqueName: \"kubernetes.io/projected/b50a2854-d80e-4a05-8215-e5f1a38b319a-kube-api-access-vwzbc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8xjnv\" (UID: \"b50a2854-d80e-4a05-8215-e5f1a38b319a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:19.834608 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.834570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwzbc\" (UniqueName: \"kubernetes.io/projected/b50a2854-d80e-4a05-8215-e5f1a38b319a-kube-api-access-vwzbc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8xjnv\" (UID: \"b50a2854-d80e-4a05-8215-e5f1a38b319a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:19.834797 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.834650 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b50a2854-d80e-4a05-8215-e5f1a38b319a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8xjnv\" (UID: \"b50a2854-d80e-4a05-8215-e5f1a38b319a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:19.834978 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.834962 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b50a2854-d80e-4a05-8215-e5f1a38b319a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8xjnv\" (UID: \"b50a2854-d80e-4a05-8215-e5f1a38b319a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:19.842887 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:19.842857 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwzbc\" (UniqueName: \"kubernetes.io/projected/b50a2854-d80e-4a05-8215-e5f1a38b319a-kube-api-access-vwzbc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8xjnv\" (UID: \"b50a2854-d80e-4a05-8215-e5f1a38b319a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:20.017811 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:20.017774 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:20.297474 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:20.297416 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv"] Sep 10 20:27:20.300224 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:27:20.300192 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50a2854_d80e_4a05_8215_e5f1a38b319a.slice/crio-2662fbfd5ad7af0ba795b864d140aacc501f8364c6cf0e676b9216a02c655764 WatchSource:0}: Error finding container 2662fbfd5ad7af0ba795b864d140aacc501f8364c6cf0e676b9216a02c655764: Status 404 returned error can't find the container with id 2662fbfd5ad7af0ba795b864d140aacc501f8364c6cf0e676b9216a02c655764 Sep 10 20:27:20.617030 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:20.616947 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" event={"ID":"b50a2854-d80e-4a05-8215-e5f1a38b319a","Type":"ContainerStarted","Data":"07647080d5dc3967b05e7da6ffc0aebcc7b7b70f31c524125c2ff035936a5be5"} Sep 10 20:27:20.617030 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:20.616996 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" event={"ID":"b50a2854-d80e-4a05-8215-e5f1a38b319a","Type":"ContainerStarted","Data":"2662fbfd5ad7af0ba795b864d140aacc501f8364c6cf0e676b9216a02c655764"} Sep 10 20:27:20.617239 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:20.617085 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:27:20.641520 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:20.641462 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" podStartSLOduration=1.641441021 podStartE2EDuration="1.641441021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:27:20.639332995 +0000 UTC m=+2009.360280117" watchObservedRunningTime="2026-09-10 20:27:20.641441021 +0000 UTC m=+2009.362388142" Sep 10 20:27:31.625163 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:27:31.625132 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8xjnv" Sep 10 20:28:52.995679 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:52.995643 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr"] Sep 10 20:28:52.999365 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:52.999345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.011736 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.011708 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 20:28:53.011884 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.011744 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 20:28:53.011884 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.011815 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 20:28:53.012117 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.012060 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q7b27\"" Sep 10 20:28:53.012302 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.012287 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 20:28:53.014373 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.014350 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr"] Sep 10 20:28:53.094927 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.094887 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-maas-api-tls\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.094927 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.094929 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-metrics-certs\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.095229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.094970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqvz\" (UniqueName: \"kubernetes.io/projected/30468dcf-756f-47c2-a392-4484512c26b2-kube-api-access-xdqvz\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.095229 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.095027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30468dcf-756f-47c2-a392-4484512c26b2-openshift-service-ca\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.195966 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.195935 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-maas-api-tls\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.195966 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.195969 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-metrics-certs\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.196262 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.196031 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqvz\" (UniqueName: \"kubernetes.io/projected/30468dcf-756f-47c2-a392-4484512c26b2-kube-api-access-xdqvz\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.196262 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.196113 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30468dcf-756f-47c2-a392-4484512c26b2-openshift-service-ca\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.196759 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.196741 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30468dcf-756f-47c2-a392-4484512c26b2-openshift-service-ca\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.198507 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.198478 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-maas-api-tls\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.198622 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.198553 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-metrics-certs\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.203902 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.203882 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqvz\" (UniqueName: \"kubernetes.io/projected/30468dcf-756f-47c2-a392-4484512c26b2-kube-api-access-xdqvz\") pod \"maas-api-7f7f8945f9-2lmwr\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.311533 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.311443 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:53.591099 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:53.591012 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr"] Sep 10 20:28:53.593565 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:28:53.593534 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30468dcf_756f_47c2_a392_4484512c26b2.slice/crio-2cb4898878de5a59ff932d22ae7bc4ac882fb0a5e0b0b9b9afc1d04483afa981 WatchSource:0}: Error finding container 2cb4898878de5a59ff932d22ae7bc4ac882fb0a5e0b0b9b9afc1d04483afa981: Status 404 returned error can't find the container with id 2cb4898878de5a59ff932d22ae7bc4ac882fb0a5e0b0b9b9afc1d04483afa981 Sep 10 20:28:54.091462 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:54.091425 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" event={"ID":"30468dcf-756f-47c2-a392-4484512c26b2","Type":"ContainerStarted","Data":"2cb4898878de5a59ff932d22ae7bc4ac882fb0a5e0b0b9b9afc1d04483afa981"} Sep 10 20:28:56.103263 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:56.103230 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" event={"ID":"30468dcf-756f-47c2-a392-4484512c26b2","Type":"ContainerStarted","Data":"7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606"} Sep 10 20:28:56.103663 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:56.103317 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:28:56.122107 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:28:56.122045 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" podStartSLOduration=2.467580639 podStartE2EDuration="4.122031849s" totalImagesPullingTime="1.65445121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:28:56.121005283 +0000 UTC m=+2104.841952416" watchObservedRunningTime="2026-09-10 20:28:56.122031849 +0000 UTC m=+2104.842978967" Sep 10 20:29:00.593725 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.593688 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr"] Sep 10 20:29:00.594248 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.593943 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" podUID="30468dcf-756f-47c2-a392-4484512c26b2" containerName="maas-api" containerID="cri-o://7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606" gracePeriod=30 Sep 10 20:29:00.600391 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.600363 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:29:00.628142 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.628116 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-f4fkv"] Sep 10 20:29:00.631619 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.631602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.644415 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.644385 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-f4fkv"] Sep 10 20:29:00.664460 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.664423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-maas-api-tls\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.664623 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.664484 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-openshift-service-ca\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.664623 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.664603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t28sg\" (UniqueName: \"kubernetes.io/projected/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-kube-api-access-t28sg\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.664717 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.664645 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-metrics-certs\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.765733 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.765697 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-maas-api-tls\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.765921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.765782 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-openshift-service-ca\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.765921 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.765887 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t28sg\" (UniqueName: \"kubernetes.io/projected/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-kube-api-access-t28sg\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.766046 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.765927 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-metrics-certs\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.766968 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.766944 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-openshift-service-ca\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.768760 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.768733 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-maas-api-tls\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.769130 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.769090 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-metrics-certs\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.774244 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.774199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t28sg\" (UniqueName: \"kubernetes.io/projected/32ee6e59-9a49-4965-8594-f0c6c5ddbb90-kube-api-access-t28sg\") pod \"maas-api-559477f47-f4fkv\" (UID: \"32ee6e59-9a49-4965-8594-f0c6c5ddbb90\") " pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:00.851679 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.851620 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:29:00.967762 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.967726 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdqvz\" (UniqueName: \"kubernetes.io/projected/30468dcf-756f-47c2-a392-4484512c26b2-kube-api-access-xdqvz\") pod \"30468dcf-756f-47c2-a392-4484512c26b2\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " Sep 10 20:29:00.967954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.967774 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-metrics-certs\") pod \"30468dcf-756f-47c2-a392-4484512c26b2\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " Sep 10 20:29:00.967954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.967816 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-maas-api-tls\") pod \"30468dcf-756f-47c2-a392-4484512c26b2\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " Sep 10 20:29:00.967954 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.967896 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30468dcf-756f-47c2-a392-4484512c26b2-openshift-service-ca\") pod \"30468dcf-756f-47c2-a392-4484512c26b2\" (UID: \"30468dcf-756f-47c2-a392-4484512c26b2\") " Sep 10 20:29:00.968346 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.968320 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30468dcf-756f-47c2-a392-4484512c26b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30468dcf-756f-47c2-a392-4484512c26b2" (UID: "30468dcf-756f-47c2-a392-4484512c26b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 20:29:00.970028 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.970005 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30468dcf-756f-47c2-a392-4484512c26b2-kube-api-access-xdqvz" (OuterVolumeSpecName: "kube-api-access-xdqvz") pod "30468dcf-756f-47c2-a392-4484512c26b2" (UID: "30468dcf-756f-47c2-a392-4484512c26b2"). InnerVolumeSpecName "kube-api-access-xdqvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 20:29:00.970154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.970047 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30468dcf-756f-47c2-a392-4484512c26b2" (UID: "30468dcf-756f-47c2-a392-4484512c26b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:29:00.970154 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.970058 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30468dcf-756f-47c2-a392-4484512c26b2" (UID: "30468dcf-756f-47c2-a392-4484512c26b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 20:29:00.974200 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:00.974183 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:01.069630 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.069595 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30468dcf-756f-47c2-a392-4484512c26b2-openshift-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:29:01.069630 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.069624 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdqvz\" (UniqueName: \"kubernetes.io/projected/30468dcf-756f-47c2-a392-4484512c26b2-kube-api-access-xdqvz\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:29:01.069630 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.069634 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-metrics-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:29:01.069864 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.069643 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30468dcf-756f-47c2-a392-4484512c26b2-maas-api-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Sep 10 20:29:01.134873 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.134792 2570 generic.go:356] "Generic (PLEG): container finished" podID="30468dcf-756f-47c2-a392-4484512c26b2" containerID="7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606" exitCode=0 Sep 10 20:29:01.134873 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.134832 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" Sep 10 20:29:01.134873 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.134843 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" event={"ID":"30468dcf-756f-47c2-a392-4484512c26b2","Type":"ContainerDied","Data":"7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606"} Sep 10 20:29:01.135138 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.134884 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr" event={"ID":"30468dcf-756f-47c2-a392-4484512c26b2","Type":"ContainerDied","Data":"2cb4898878de5a59ff932d22ae7bc4ac882fb0a5e0b0b9b9afc1d04483afa981"} Sep 10 20:29:01.135138 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.134905 2570 scope.go:117] "RemoveContainer" containerID="7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606" Sep 10 20:29:01.145924 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.145908 2570 scope.go:117] "RemoveContainer" containerID="7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606" Sep 10 20:29:01.146224 ip-10-0-130-59 kubenswrapper[2570]: E0910 20:29:01.146206 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606\": container with ID starting with 7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606 not found: ID does not exist" containerID="7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606" Sep 10 20:29:01.146288 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.146235 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606"} err="failed to get container status \"7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606\": rpc error: code = NotFound desc = could not find container \"7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606\": container with ID starting with 7300eb706840aee6a19ef7bb12c5814682a73eb84d6d3a6ee2549dd32e113606 not found: ID does not exist" Sep 10 20:29:01.161809 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.161785 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr"] Sep 10 20:29:01.165513 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.165490 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-2lmwr"] Sep 10 20:29:01.247482 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.247451 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-f4fkv"] Sep 10 20:29:01.248210 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:29:01.248184 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ee6e59_9a49_4965_8594_f0c6c5ddbb90.slice/crio-2a848c85ee30abc8d2274e809a66eea6c8b6bcaf5950364838e604ac48372c30 WatchSource:0}: Error finding container 2a848c85ee30abc8d2274e809a66eea6c8b6bcaf5950364838e604ac48372c30: Status 404 returned error can't find the container with id 2a848c85ee30abc8d2274e809a66eea6c8b6bcaf5950364838e604ac48372c30 Sep 10 20:29:01.765605 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:01.765571 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30468dcf-756f-47c2-a392-4484512c26b2" path="/var/lib/kubelet/pods/30468dcf-756f-47c2-a392-4484512c26b2/volumes" Sep 10 20:29:02.142064 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:02.142030 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" event={"ID":"32ee6e59-9a49-4965-8594-f0c6c5ddbb90","Type":"ContainerStarted","Data":"0614cc4a898f0f18c8fb89478e38e9d971c86f481d593b0ae15e6e2ba1a36fda"} Sep 10 20:29:02.142281 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:02.142119 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:02.142281 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:02.142141 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" event={"ID":"32ee6e59-9a49-4965-8594-f0c6c5ddbb90","Type":"ContainerStarted","Data":"2a848c85ee30abc8d2274e809a66eea6c8b6bcaf5950364838e604ac48372c30"} Sep 10 20:29:02.160066 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:02.160011 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" podStartSLOduration=1.789132689 podStartE2EDuration="2.159996552s" totalImagesPullingTime="370.863863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:29:02.158891301 +0000 UTC m=+2110.879838421" watchObservedRunningTime="2026-09-10 20:29:02.159996552 +0000 UTC m=+2110.880943672" Sep 10 20:29:08.154255 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:08.154226 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-559477f47-f4fkv" Sep 10 20:29:28.702685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:28.702652 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fc475458-56m2z_336e0226-37c2-4843-a45a-64e6550a7e86/manager/0.log" Sep 10 20:29:29.339583 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:29.339551 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5678c4c4c5-s6zwm_1a0eb861-1be8-455d-ae15-092de7e963d7/manager/0.log" Sep 10 20:29:29.346868 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:29.346843 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5678c4c4c5-s6zwm_1a0eb861-1be8-455d-ae15-092de7e963d7/copy-manifests/0.log" Sep 10 20:29:29.701912 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:29.701824 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-559477f47-f4fkv_32ee6e59-9a49-4965-8594-f0c6c5ddbb90/maas-api/0.log" Sep 10 20:29:29.940399 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:29.940361 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817855-j7nd2_125db6f3-d1bc-4f42-99b1-8f9cda9bf510/cleanup/0.log" Sep 10 20:29:31.349292 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:31.349258 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-8xjnv_b50a2854-d80e-4a05-8215-e5f1a38b319a/manager/0.log" Sep 10 20:29:31.938179 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:31.938131 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt_ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385/istio-proxy/0.log" Sep 10 20:29:32.063223 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:32.063193 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l9w4g_2e92d525-ff7c-46d2-b1e8-861d0b37db1e/discovery/0.log" Sep 10 20:29:32.657094 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:32.657027 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6v58f_3e8f86ab-11fb-41d8-97e2-0b1077ea1e03/payload-processing/0.log" Sep 10 20:29:33.146151 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:33.146116 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4hmxf_18b97df0-df1c-41fc-9f06-ec00a01bb114/main/0.log" Sep 10 20:29:33.383876 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:33.383841 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-lvwrl_e40b7da6-5e82-4d1a-b44a-04098ba75439/main/0.log" Sep 10 20:29:43.040305 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:43.040267 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-flfcj_c2cd9622-ed11-4e4c-b30a-7d94dc097090/global-pull-secret-syncer/0.log" Sep 10 20:29:43.251983 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:43.251935 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gxpgv_2450cbac-1850-4128-9a10-68d6d582c549/konnectivity-agent/0.log" Sep 10 20:29:43.341223 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:43.341131 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-59.ec2.internal_48adfbd41af55e45ba98e8777ebfc093/haproxy/0.log" Sep 10 20:29:48.568885 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:48.568855 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-8xjnv_b50a2854-d80e-4a05-8215-e5f1a38b319a/manager/0.log" Sep 10 20:29:50.326539 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.326448 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/alertmanager/0.log" Sep 10 20:29:50.353807 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.353775 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/config-reloader/0.log" Sep 10 20:29:50.378833 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.378781 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/kube-rbac-proxy-web/0.log" Sep 10 20:29:50.405794 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.405763 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/kube-rbac-proxy/0.log" Sep 10 20:29:50.443027 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.442990 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/kube-rbac-proxy-metric/0.log" Sep 10 20:29:50.471018 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.470991 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/prom-label-proxy/0.log" Sep 10 20:29:50.497641 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.497602 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e5fd52e1-96bc-4169-84c5-a61d15d40e04/init-config-reloader/0.log" Sep 10 20:29:50.591916 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.591891 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-d248x_107b8bf9-9f05-469b-ac9c-9dc8e7b24f28/kube-state-metrics/0.log" Sep 10 20:29:50.627642 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.627615 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-d248x_107b8bf9-9f05-469b-ac9c-9dc8e7b24f28/kube-rbac-proxy-main/0.log" Sep 10 20:29:50.662234 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.662206 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-d248x_107b8bf9-9f05-469b-ac9c-9dc8e7b24f28/kube-rbac-proxy-self/0.log" Sep 10 20:29:50.734365 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.734337 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-mlxnr_342f12a4-faa4-432f-a282-19fe22752cac/monitoring-plugin/0.log" Sep 10 20:29:50.770307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.770273 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7xgb9_1c955aee-1ab4-474e-86c3-bd3f46776c8e/node-exporter/0.log" Sep 10 20:29:50.799885 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.799860 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7xgb9_1c955aee-1ab4-474e-86c3-bd3f46776c8e/kube-rbac-proxy/0.log" Sep 10 20:29:50.825182 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:50.825158 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7xgb9_1c955aee-1ab4-474e-86c3-bd3f46776c8e/init-textfile/0.log" Sep 10 20:29:51.024563 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.024537 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-tmsrh_5d701640-ef43-4707-8982-eb3aceaba4b6/kube-rbac-proxy-main/0.log" Sep 10 20:29:51.052307 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.052282 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-tmsrh_5d701640-ef43-4707-8982-eb3aceaba4b6/kube-rbac-proxy-self/0.log" Sep 10 20:29:51.073119 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.073093 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-tmsrh_5d701640-ef43-4707-8982-eb3aceaba4b6/openshift-state-metrics/0.log" Sep 10 20:29:51.111974 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.111949 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/prometheus/0.log" Sep 10 20:29:51.138811 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.138790 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/config-reloader/0.log" Sep 10 20:29:51.165105 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.165060 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/thanos-sidecar/0.log" Sep 10 20:29:51.192717 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.192651 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/kube-rbac-proxy-web/0.log" Sep 10 20:29:51.218248 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.218228 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/kube-rbac-proxy/0.log" Sep 10 20:29:51.242817 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.242795 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/kube-rbac-proxy-thanos/0.log" Sep 10 20:29:51.265224 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.265205 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_58deb00d-8ebe-4872-8713-425e6aa0f10e/init-config-reloader/0.log" Sep 10 20:29:51.349234 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.349206 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-g9zg6_a236573e-f358-4fdf-aba3-f3b067e5473e/prometheus-operator-admission-webhook/0.log" Sep 10 20:29:51.379028 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.378995 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c94f4cbbf-2mlsc_dddec83d-7d0e-4692-9338-25b97ef8b445/telemeter-client/0.log" Sep 10 20:29:51.410926 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.410898 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c94f4cbbf-2mlsc_dddec83d-7d0e-4692-9338-25b97ef8b445/reload/0.log" Sep 10 20:29:51.431360 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:51.431337 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c94f4cbbf-2mlsc_dddec83d-7d0e-4692-9338-25b97ef8b445/kube-rbac-proxy/0.log" Sep 10 20:29:52.232937 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:52.232903 2570 scope.go:117] "RemoveContainer" containerID="9a799c3ffecfdcdf88075dff4ed4627ea79812e922bff931df34b9323e1d5354" Sep 10 20:29:52.848185 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:52.848151 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-6l6vv_84bd2c76-eb69-4f97-9662-b8b1272599f8/networking-console-plugin/0.log" Sep 10 20:29:53.104577 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.104486 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz"] Sep 10 20:29:53.104967 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.104952 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30468dcf-756f-47c2-a392-4484512c26b2" containerName="maas-api" Sep 10 20:29:53.105012 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.104968 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="30468dcf-756f-47c2-a392-4484512c26b2" containerName="maas-api" Sep 10 20:29:53.105056 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.105047 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="30468dcf-756f-47c2-a392-4484512c26b2" containerName="maas-api" Sep 10 20:29:53.109805 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.109783 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.112195 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.112175 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g28c4\"/\"openshift-service-ca.crt\"" Sep 10 20:29:53.112994 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.112976 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g28c4\"/\"default-dockercfg-qnx6f\"" Sep 10 20:29:53.113117 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.113016 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g28c4\"/\"kube-root-ca.crt\"" Sep 10 20:29:53.119183 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.119161 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz"] Sep 10 20:29:53.276785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.276747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-podres\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.276785 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.276789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-lib-modules\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.277007 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.276825 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqkx2\" (UniqueName: \"kubernetes.io/projected/b4cd406f-b51a-4bda-9966-23799ed13bee-kube-api-access-rqkx2\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.277007 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.276909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-proc\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.277007 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.276933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-sys\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378116 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-proc\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-sys\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378207 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-podres\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378219 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-lib-modules\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378254 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-proc\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378258 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-sys\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378260 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqkx2\" (UniqueName: \"kubernetes.io/projected/b4cd406f-b51a-4bda-9966-23799ed13bee-kube-api-access-rqkx2\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378339 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-lib-modules\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.378472 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.378344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4cd406f-b51a-4bda-9966-23799ed13bee-podres\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.387845 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.387815 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqkx2\" (UniqueName: \"kubernetes.io/projected/b4cd406f-b51a-4bda-9966-23799ed13bee-kube-api-access-rqkx2\") pod \"perf-node-gather-daemonset-fdwgz\" (UID: \"b4cd406f-b51a-4bda-9966-23799ed13bee\") " pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.421268 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.421240 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:53.698488 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:53.698464 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz"] Sep 10 20:29:53.700764 ip-10-0-130-59 kubenswrapper[2570]: W0910 20:29:53.700739 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb4cd406f_b51a_4bda_9966_23799ed13bee.slice/crio-16eeb5eef4c9904fff91ffdad1f5a85cc98e717677b1c952eb191605ba1a0543 WatchSource:0}: Error finding container 16eeb5eef4c9904fff91ffdad1f5a85cc98e717677b1c952eb191605ba1a0543: Status 404 returned error can't find the container with id 16eeb5eef4c9904fff91ffdad1f5a85cc98e717677b1c952eb191605ba1a0543 Sep 10 20:29:54.198826 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:54.198790 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-5w6xl_6f7f7aff-7891-47be-b85a-0c30ead85c10/download-server/0.log" Sep 10 20:29:54.402380 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:54.402348 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" event={"ID":"b4cd406f-b51a-4bda-9966-23799ed13bee","Type":"ContainerStarted","Data":"c0127e35e430b329279daf8e3d12629780542c0d531f75ea95f670d7c6164e9d"} Sep 10 20:29:54.402536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:54.402406 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" event={"ID":"b4cd406f-b51a-4bda-9966-23799ed13bee","Type":"ContainerStarted","Data":"16eeb5eef4c9904fff91ffdad1f5a85cc98e717677b1c952eb191605ba1a0543"} Sep 10 20:29:54.402536 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:54.402434 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:29:54.422714 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:54.422658 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" podStartSLOduration=1.422642532 podStartE2EDuration="1.422642532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:29:54.421855687 +0000 UTC m=+2163.142802809" watchObservedRunningTime="2026-09-10 20:29:54.422642532 +0000 UTC m=+2163.143589704" Sep 10 20:29:54.904890 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:54.904858 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-mm2w9_acdd628d-de21-405b-9683-b0b6cc4de4d3/volume-data-source-validator/0.log" Sep 10 20:29:56.161324 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:56.161294 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tfpxv_6e92499b-cb2e-4203-8247-f3ca5d908d4e/dns/0.log" Sep 10 20:29:56.180512 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:56.180489 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tfpxv_6e92499b-cb2e-4203-8247-f3ca5d908d4e/kube-rbac-proxy/0.log" Sep 10 20:29:56.299923 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:56.299895 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tpt2k_1a17871d-91b4-42ed-90ae-2feee6eb985a/dns-node-resolver/0.log" Sep 10 20:29:56.897213 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:56.897184 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dspv5_1c2a39fa-8215-4571-aa00-cdcd08fc3087/node-ca/0.log" Sep 10 20:29:57.900181 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:57.900155 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt_ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385/istio-proxy/0.log" Sep 10 20:29:58.328583 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:58.328548 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l9w4g_2e92d525-ff7c-46d2-b1e8-861d0b37db1e/discovery/0.log" Sep 10 20:29:58.499451 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:58.499415 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6v58f_3e8f86ab-11fb-41d8-97e2-0b1077ea1e03/payload-processing/0.log" Sep 10 20:29:59.439808 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:29:59.439780 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-glzkt_a0a7eead-c04c-4105-82b5-d060fe76f2ef/serve-healthcheck-canary/0.log" Sep 10 20:30:00.029404 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:00.029375 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4hgsm_b53748e6-5b90-4951-8154-cac98ee3d801/kube-rbac-proxy/0.log" Sep 10 20:30:00.054747 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:00.054725 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4hgsm_b53748e6-5b90-4951-8154-cac98ee3d801/exporter/0.log" Sep 10 20:30:00.078929 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:00.078897 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4hgsm_b53748e6-5b90-4951-8154-cac98ee3d801/extractor/0.log" Sep 10 20:30:00.417535 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:00.417465 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g28c4/perf-node-gather-daemonset-fdwgz" Sep 10 20:30:03.123685 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:03.123641 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fc475458-56m2z_336e0226-37c2-4843-a45a-64e6550a7e86/manager/0.log" Sep 10 20:30:03.693913 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:03.693880 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5678c4c4c5-s6zwm_1a0eb861-1be8-455d-ae15-092de7e963d7/manager/0.log" Sep 10 20:30:03.741509 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:03.741481 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5678c4c4c5-s6zwm_1a0eb861-1be8-455d-ae15-092de7e963d7/copy-manifests/0.log" Sep 10 20:30:04.231658 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:04.231619 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt_ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385/istio-proxy/0.log" Sep 10 20:30:04.643666 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:04.643628 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l9w4g_2e92d525-ff7c-46d2-b1e8-861d0b37db1e/discovery/0.log" Sep 10 20:30:04.825920 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:04.825882 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6v58f_3e8f86ab-11fb-41d8-97e2-0b1077ea1e03/payload-processing/0.log" Sep 10 20:30:05.055241 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:05.055213 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6fd976b7bd-pn47n_22a9b59a-243d-4dc2-ad00-f73486afd675/manager/0.log" Sep 10 20:30:06.383057 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:06.383026 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cwvnjt_ab17c8e0-4c7d-4cbd-8b93-7eb356dd4385/istio-proxy/0.log" Sep 10 20:30:06.814247 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:06.814200 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l9w4g_2e92d525-ff7c-46d2-b1e8-861d0b37db1e/discovery/0.log" Sep 10 20:30:06.960267 ip-10-0-130-59 kubenswrapper[2570]: I0910 20:30:06.960224 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6v58f_3e8f86ab-11fb-41d8-97e2-0b1077ea1e03/payload-processing/0.log"